Note: This is test shard 3 of 6.
[==========] Running 7 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from RaftConsensusITest
[ RUN      ] RaftConsensusITest.TestFailedOp
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260812 08:00:51.287411  5690 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:00:51.287556  5690 ts_itest-base.cc:116] --------------
I20260812 08:00:51.287567  5690 ts_itest-base.cc:117] 3 tablet servers
I20260812 08:00:51.287578  5690 ts_itest-base.cc:118] 3 replicas per TS
I20260812 08:00:51.287588  5690 ts_itest-base.cc:119] --------------
2026-08-12T08:00:51Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:00:51Z Disabled control of system clock
I20260812 08:00:51.301888  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.142.190:38049
--webserver_interface=127.5.142.190
--webserver_port=0
--builtin_ntp_servers=127.5.142.148:42237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.142.190:38049 with env {}
W20260812 08:00:51.419872  5705 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:51.420089  5705 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:51.420127  5705 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:51.423714  5705 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:00:51.423801  5705 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:51.423826  5705 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:00:51.423856  5705 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:00:51.428061  5705 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:42237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.142.190:38049
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.142.190:38049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.5.142.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.5705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:51.429363  5705 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:51.430757  5705 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:51.437093  5716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:51.437124  5712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:51.437229  5705 server_base.cc:1061] running on GCE node
W20260812 08:00:51.437155  5713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:51.437762  5705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:51.438743  5705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:51.439924  5705 hybrid_clock.cc:648] HybridClock initialized: now 1786521651439895 us; error 43 us; skew 500 ppm
I20260812 08:00:51.441848  5705 webserver.cc:533] Webserver started at http://127.5.142.190:41159/ using document root <none> and password file <none>
I20260812 08:00:51.442437  5705 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:51.442499  5705 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:51.442714  5705 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:51.444321  5705 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "28e2d3640dd24347b929e312c6440afa"
format_stamp: "Formatted at 2026-08-12 08:00:51 on dist-test-slave-ncp9"
I20260812 08:00:51.444979  5705 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "28e2d3640dd24347b929e312c6440afa"
format_stamp: "Formatted at 2026-08-12 08:00:51 on dist-test-slave-ncp9"
I20260812 08:00:51.449404  5705 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.001s
I20260812 08:00:51.451997  5728 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:51.453125  5705 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:00:51.453262  5705 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
uuid: "28e2d3640dd24347b929e312c6440afa"
format_stamp: "Formatted at 2026-08-12 08:00:51 on dist-test-slave-ncp9"
I20260812 08:00:51.453359  5705 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:51.485015  5705 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:51.485847  5705 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:51.486078  5705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:51.495386  5825 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.190:38049 every 8 connection(s)
I20260812 08:00:51.495401  5705 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.190:38049
I20260812 08:00:51.496610  5705 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:00:51.499464  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 5705
I20260812 08:00:51.499619  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:00:51.500424  5826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:51.506961  5826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28e2d3640dd24347b929e312c6440afa: Bootstrap starting.
I20260812 08:00:51.509570  5826 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28e2d3640dd24347b929e312c6440afa: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:51.510329  5826 log.cc:826] T 00000000000000000000000000000000 P 28e2d3640dd24347b929e312c6440afa: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:51.512611  5826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28e2d3640dd24347b929e312c6440afa: No bootstrap required, opened a new log
I20260812 08:00:51.515323  5826 raft_consensus.cc:359] T 00000000000000000000000000000000 P 28e2d3640dd24347b929e312c6440afa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28e2d3640dd24347b929e312c6440afa" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 38049 } }
I20260812 08:00:51.515609  5826 raft_consensus.cc:385] T 00000000000000000000000000000000 P 28e2d3640dd24347b929e312c6440afa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:51.515683  5826 raft_consensus.cc:740] T 00000000000000000000000000000000 P 28e2d3640dd24347b929e312c6440afa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28e2d3640dd24347b929e312c6440afa, State: Initialized, Role: FOLLOWER
I20260812 08:00:51.516237  5826 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28e2d3640dd24347b929e312c6440afa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28e2d3640dd24347b929e312c6440afa" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 38049 } }
I20260812 08:00:51.516417  5826 raft_consensus.cc:399] T 00000000000000000000000000000000 P 28e2d3640dd24347b929e312c6440afa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:00:51.516535  5826 raft_consensus.cc:493] T 00000000000000000000000000000000 P 28e2d3640dd24347b929e312c6440afa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:00:51.516649  5826 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 28e2d3640dd24347b929e312c6440afa [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:51.517644  5826 raft_consensus.cc:515] T 00000000000000000000000000000000 P 28e2d3640dd24347b929e312c6440afa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28e2d3640dd24347b929e312c6440afa" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 38049 } }
I20260812 08:00:51.518016  5826 leader_election.cc:304] T 00000000000000000000000000000000 P 28e2d3640dd24347b929e312c6440afa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28e2d3640dd24347b929e312c6440afa; no voters: 
I20260812 08:00:51.518353  5826 leader_election.cc:290] T 00000000000000000000000000000000 P 28e2d3640dd24347b929e312c6440afa [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:00:51.518443  5833 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 28e2d3640dd24347b929e312c6440afa [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:00:51.518599  5833 raft_consensus.cc:697] T 00000000000000000000000000000000 P 28e2d3640dd24347b929e312c6440afa [term 1 LEADER]: Becoming Leader. State: Replica: 28e2d3640dd24347b929e312c6440afa, State: Running, Role: LEADER
I20260812 08:00:51.519049  5833 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28e2d3640dd24347b929e312c6440afa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28e2d3640dd24347b929e312c6440afa" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 38049 } }
I20260812 08:00:51.519790  5826 sys_catalog.cc:565] T 00000000000000000000000000000000 P 28e2d3640dd24347b929e312c6440afa [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:00:51.520633  5833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28e2d3640dd24347b929e312c6440afa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28e2d3640dd24347b929e312c6440afa. Latest consensus state: current_term: 1 leader_uuid: "28e2d3640dd24347b929e312c6440afa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28e2d3640dd24347b929e312c6440afa" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 38049 } } }
I20260812 08:00:51.520769  5833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28e2d3640dd24347b929e312c6440afa [sys.catalog]: This master's current role is: LEADER
I20260812 08:00:51.521890  5848 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:00:51.521904  5836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28e2d3640dd24347b929e312c6440afa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28e2d3640dd24347b929e312c6440afa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28e2d3640dd24347b929e312c6440afa" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 38049 } } }
I20260812 08:00:51.522034  5836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28e2d3640dd24347b929e312c6440afa [sys.catalog]: This master's current role is: LEADER
I20260812 08:00:51.525130  5848 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:00:51.530694  5848 catalog_manager.cc:1383] Generated new cluster ID: 011ebca2cd8c424b9cf08620d11f5dfe
I20260812 08:00:51.530764  5848 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:00:51.542941  5848 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:00:51.543768  5848 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:00:51.559156  5848 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 28e2d3640dd24347b929e312c6440afa: Generated new TSK 0
I20260812 08:00:51.559811  5848 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:00:51.569415  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.129:0
--local_ip_for_outbound_sockets=127.5.142.129
--webserver_interface=127.5.142.129
--webserver_port=0
--tserver_master_addrs=127.5.142.190:38049
--builtin_ntp_servers=127.5.142.148:42237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=50 with env {}
W20260812 08:00:51.694239  5876 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:51.694470  5876 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:51.694521  5876 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:51.697904  5876 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:51.698027  5876 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.129
I20260812 08:00:51.701972  5876 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:42237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=50
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.5.142.129
--webserver_port=0
--tserver_master_addrs=127.5.142.190:38049
--never_fsync=true
--heap_profile_path=/tmp/kudu.5876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.129

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:51.703208  5876 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:51.704407  5876 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:51.711668  5883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:51.711658  5885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:51.711663  5887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:51.711884  5876 server_base.cc:1061] running on GCE node
I20260812 08:00:51.712386  5876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:51.712990  5876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:51.714145  5876 hybrid_clock.cc:648] HybridClock initialized: now 1786521651714121 us; error 39 us; skew 500 ppm
I20260812 08:00:51.715920  5876 webserver.cc:533] Webserver started at http://127.5.142.129:36751/ using document root <none> and password file <none>
I20260812 08:00:51.716501  5876 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:51.716558  5876 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:51.716761  5876 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:51.718389  5876 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "88401325678c4312afd3853dd06928c1"
format_stamp: "Formatted at 2026-08-12 08:00:51 on dist-test-slave-ncp9"
I20260812 08:00:51.718832  5876 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "88401325678c4312afd3853dd06928c1"
format_stamp: "Formatted at 2026-08-12 08:00:51 on dist-test-slave-ncp9"
I20260812 08:00:51.722388  5876 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:00:51.724646  5897 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:51.725704  5876 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:00:51.725801  5876 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "88401325678c4312afd3853dd06928c1"
format_stamp: "Formatted at 2026-08-12 08:00:51 on dist-test-slave-ncp9"
I20260812 08:00:51.725883  5876 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:51.740314  5876 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:51.741137  5876 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:51.741312  5876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:51.741950  5876 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:51.743141  5876 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:51.743197  5876 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:51.743242  5876 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:51.743263  5876 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:51.755965  5876 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.129:41357
I20260812 08:00:51.755988  6097 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.129:41357 every 8 connection(s)
I20260812 08:00:51.757102  5876 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:00:51.757647  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 5876
I20260812 08:00:51.757741  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:00:51.760272  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.130:0
--local_ip_for_outbound_sockets=127.5.142.130
--webserver_interface=127.5.142.130
--webserver_port=0
--tserver_master_addrs=127.5.142.190:38049
--builtin_ntp_servers=127.5.142.148:42237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=50 with env {}
I20260812 08:00:51.766819  6098 heartbeater.cc:344] Connected to a master server at 127.5.142.190:38049
I20260812 08:00:51.767102  6098 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:51.767807  6098 heartbeater.cc:507] Master 127.5.142.190:38049 requested a full tablet report, sending...
I20260812 08:00:51.769790  5753 ts_manager.cc:194] Registered new tserver with Master: 88401325678c4312afd3853dd06928c1 (127.5.142.129:41357)
I20260812 08:00:51.771450  5753 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.129:44187
W20260812 08:00:51.874310  6102 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:51.874540  6102 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:51.874583  6102 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:51.877938  6102 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:51.878047  6102 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.130
I20260812 08:00:51.881910  6102 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:42237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=50
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.5.142.130
--webserver_port=0
--tserver_master_addrs=127.5.142.190:38049
--never_fsync=true
--heap_profile_path=/tmp/kudu.6102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.130

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:51.883198  6102 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:51.884382  6102 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:51.891008  6110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:51.891002  6113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:51.891227  6102 server_base.cc:1061] running on GCE node
W20260812 08:00:51.891376  6111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:51.891741  6102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:51.892278  6102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:51.893429  6102 hybrid_clock.cc:648] HybridClock initialized: now 1786521651893409 us; error 28 us; skew 500 ppm
I20260812 08:00:51.895388  6102 webserver.cc:533] Webserver started at http://127.5.142.130:42757/ using document root <none> and password file <none>
I20260812 08:00:51.895985  6102 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:51.896045  6102 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:51.896234  6102 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:51.897894  6102 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "cf512a43dd17478a9ef9e1c1c34d98a1"
format_stamp: "Formatted at 2026-08-12 08:00:51 on dist-test-slave-ncp9"
I20260812 08:00:51.898382  6102 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "cf512a43dd17478a9ef9e1c1c34d98a1"
format_stamp: "Formatted at 2026-08-12 08:00:51 on dist-test-slave-ncp9"
I20260812 08:00:51.901980  6102 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.000s	sys 0.004s
I20260812 08:00:51.904181  6131 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:51.905192  6102 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:00:51.905299  6102 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "cf512a43dd17478a9ef9e1c1c34d98a1"
format_stamp: "Formatted at 2026-08-12 08:00:51 on dist-test-slave-ncp9"
I20260812 08:00:51.905385  6102 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:51.921924  6102 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:51.922549  6102 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:51.922719  6102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:51.923282  6102 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:51.924376  6102 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:51.924425  6102 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:51.924480  6102 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:51.924507  6102 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:51.936327  6102 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.130:45103
I20260812 08:00:51.936347  6326 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.130:45103 every 8 connection(s)
I20260812 08:00:51.937386  6102 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:00:51.937775  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 6102
I20260812 08:00:51.937877  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:00:51.940544  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.131:0
--local_ip_for_outbound_sockets=127.5.142.131
--webserver_interface=127.5.142.131
--webserver_port=0
--tserver_master_addrs=127.5.142.190:38049
--builtin_ntp_servers=127.5.142.148:42237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=50 with env {}
I20260812 08:00:51.946918  6327 heartbeater.cc:344] Connected to a master server at 127.5.142.190:38049
I20260812 08:00:51.947391  6327 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:51.948180  6327 heartbeater.cc:507] Master 127.5.142.190:38049 requested a full tablet report, sending...
I20260812 08:00:51.949513  5753 ts_manager.cc:194] Registered new tserver with Master: cf512a43dd17478a9ef9e1c1c34d98a1 (127.5.142.130:45103)
I20260812 08:00:51.950116  5753 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.130:42517
W20260812 08:00:52.049948  6337 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:52.050143  6337 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:52.050184  6337 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:52.053436  6337 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:52.053540  6337 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.131
I20260812 08:00:52.057269  6337 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:42237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=50
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.5.142.131
--webserver_port=0
--tserver_master_addrs=127.5.142.190:38049
--never_fsync=true
--heap_profile_path=/tmp/kudu.6337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.131

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:52.058363  6337 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:52.059589  6337 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:52.066126  6348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:52.066138  6346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:52.066177  6352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:52.066362  6337 server_base.cc:1061] running on GCE node
I20260812 08:00:52.066836  6337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:52.067458  6337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:52.068608  6337 hybrid_clock.cc:648] HybridClock initialized: now 1786521652068591 us; error 30 us; skew 500 ppm
I20260812 08:00:52.070434  6337 webserver.cc:533] Webserver started at http://127.5.142.131:35691/ using document root <none> and password file <none>
I20260812 08:00:52.071004  6337 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:52.071064  6337 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:52.071251  6337 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:52.072850  6337 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "92d9c00c6a8c4679aaf96e4cb6586cbc"
format_stamp: "Formatted at 2026-08-12 08:00:52 on dist-test-slave-ncp9"
I20260812 08:00:52.073408  6337 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "92d9c00c6a8c4679aaf96e4cb6586cbc"
format_stamp: "Formatted at 2026-08-12 08:00:52 on dist-test-slave-ncp9"
I20260812 08:00:52.076630  6337 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:00:52.078742  6362 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:52.079751  6337 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:00:52.079857  6337 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "92d9c00c6a8c4679aaf96e4cb6586cbc"
format_stamp: "Formatted at 2026-08-12 08:00:52 on dist-test-slave-ncp9"
I20260812 08:00:52.079943  6337 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:52.104020  6337 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:52.104792  6337 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:52.105018  6337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:52.105654  6337 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:52.106743  6337 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:52.106794  6337 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:52.106849  6337 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:52.106873  6337 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:52.120184  6337 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.131:44745
I20260812 08:00:52.120203  6564 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.131:44745 every 8 connection(s)
I20260812 08:00:52.121292  6337 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:00:52.128324  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 6337
I20260812 08:00:52.128438  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:00:52.130693  6568 heartbeater.cc:344] Connected to a master server at 127.5.142.190:38049
I20260812 08:00:52.130973  6568 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:52.131534  6568 heartbeater.cc:507] Master 127.5.142.190:38049 requested a full tablet report, sending...
I20260812 08:00:52.132679  5753 ts_manager.cc:194] Registered new tserver with Master: 92d9c00c6a8c4679aaf96e4cb6586cbc (127.5.142.131:44745)
I20260812 08:00:52.133386  5753 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.131:58141
I20260812 08:00:52.144032  5690 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:00:52.158921  5753 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59174:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
W20260812 08:00:52.161693  5753 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 08:00:52.179759  6436 tablet_service.cc:1511] Processing CreateTablet for tablet ef4d86b0a9a64f1192ee5c577de91268 (DEFAULT_TABLE table=TestTable [id=16bc82c2ac4d49e3a7bf3f2d5dab984d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:52.179759  5982 tablet_service.cc:1511] Processing CreateTablet for tablet ef4d86b0a9a64f1192ee5c577de91268 (DEFAULT_TABLE table=TestTable [id=16bc82c2ac4d49e3a7bf3f2d5dab984d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:52.180858  5982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef4d86b0a9a64f1192ee5c577de91268. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:52.180858  6436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef4d86b0a9a64f1192ee5c577de91268. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:52.181433  6212 tablet_service.cc:1511] Processing CreateTablet for tablet ef4d86b0a9a64f1192ee5c577de91268 (DEFAULT_TABLE table=TestTable [id=16bc82c2ac4d49e3a7bf3f2d5dab984d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:52.182318  6212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef4d86b0a9a64f1192ee5c577de91268. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:52.187690  6597 tablet_bootstrap.cc:492] T ef4d86b0a9a64f1192ee5c577de91268 P cf512a43dd17478a9ef9e1c1c34d98a1: Bootstrap starting.
I20260812 08:00:52.187988  6595 tablet_bootstrap.cc:492] T ef4d86b0a9a64f1192ee5c577de91268 P 92d9c00c6a8c4679aaf96e4cb6586cbc: Bootstrap starting.
I20260812 08:00:52.188879  6596 tablet_bootstrap.cc:492] T ef4d86b0a9a64f1192ee5c577de91268 P 88401325678c4312afd3853dd06928c1: Bootstrap starting.
I20260812 08:00:52.189929  6597 tablet_bootstrap.cc:654] T ef4d86b0a9a64f1192ee5c577de91268 P cf512a43dd17478a9ef9e1c1c34d98a1: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:52.190207  6595 tablet_bootstrap.cc:654] T ef4d86b0a9a64f1192ee5c577de91268 P 92d9c00c6a8c4679aaf96e4cb6586cbc: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:52.190677  6597 log.cc:826] T ef4d86b0a9a64f1192ee5c577de91268 P cf512a43dd17478a9ef9e1c1c34d98a1: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:52.191018  6595 log.cc:826] T ef4d86b0a9a64f1192ee5c577de91268 P 92d9c00c6a8c4679aaf96e4cb6586cbc: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:52.191097  6596 tablet_bootstrap.cc:654] T ef4d86b0a9a64f1192ee5c577de91268 P 88401325678c4312afd3853dd06928c1: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:52.191936  6596 log.cc:826] T ef4d86b0a9a64f1192ee5c577de91268 P 88401325678c4312afd3853dd06928c1: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:52.192813  6597 tablet_bootstrap.cc:492] T ef4d86b0a9a64f1192ee5c577de91268 P cf512a43dd17478a9ef9e1c1c34d98a1: No bootstrap required, opened a new log
I20260812 08:00:52.192847  6595 tablet_bootstrap.cc:492] T ef4d86b0a9a64f1192ee5c577de91268 P 92d9c00c6a8c4679aaf96e4cb6586cbc: No bootstrap required, opened a new log
I20260812 08:00:52.193053  6595 ts_tablet_manager.cc:1403] T ef4d86b0a9a64f1192ee5c577de91268 P 92d9c00c6a8c4679aaf96e4cb6586cbc: Time spent bootstrapping tablet: real 0.005s	user 0.001s	sys 0.004s
I20260812 08:00:52.193048  6597 ts_tablet_manager.cc:1403] T ef4d86b0a9a64f1192ee5c577de91268 P cf512a43dd17478a9ef9e1c1c34d98a1: Time spent bootstrapping tablet: real 0.006s	user 0.005s	sys 0.000s
I20260812 08:00:52.193631  6596 tablet_bootstrap.cc:492] T ef4d86b0a9a64f1192ee5c577de91268 P 88401325678c4312afd3853dd06928c1: No bootstrap required, opened a new log
I20260812 08:00:52.193809  6596 ts_tablet_manager.cc:1403] T ef4d86b0a9a64f1192ee5c577de91268 P 88401325678c4312afd3853dd06928c1: Time spent bootstrapping tablet: real 0.005s	user 0.000s	sys 0.004s
I20260812 08:00:52.195777  6595 raft_consensus.cc:359] T ef4d86b0a9a64f1192ee5c577de91268 P 92d9c00c6a8c4679aaf96e4cb6586cbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88401325678c4312afd3853dd06928c1" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41357 } } peers { permanent_uuid: "cf512a43dd17478a9ef9e1c1c34d98a1" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 45103 } } peers { permanent_uuid: "92d9c00c6a8c4679aaf96e4cb6586cbc" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 44745 } }
I20260812 08:00:52.195964  6595 raft_consensus.cc:385] T ef4d86b0a9a64f1192ee5c577de91268 P 92d9c00c6a8c4679aaf96e4cb6586cbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:52.196039  6595 raft_consensus.cc:740] T ef4d86b0a9a64f1192ee5c577de91268 P 92d9c00c6a8c4679aaf96e4cb6586cbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92d9c00c6a8c4679aaf96e4cb6586cbc, State: Initialized, Role: FOLLOWER
I20260812 08:00:52.196084  6597 raft_consensus.cc:359] T ef4d86b0a9a64f1192ee5c577de91268 P cf512a43dd17478a9ef9e1c1c34d98a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88401325678c4312afd3853dd06928c1" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41357 } } peers { permanent_uuid: "cf512a43dd17478a9ef9e1c1c34d98a1" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 45103 } } peers { permanent_uuid: "92d9c00c6a8c4679aaf96e4cb6586cbc" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 44745 } }
I20260812 08:00:52.196323  6597 raft_consensus.cc:385] T ef4d86b0a9a64f1192ee5c577de91268 P cf512a43dd17478a9ef9e1c1c34d98a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:52.196390  6597 raft_consensus.cc:740] T ef4d86b0a9a64f1192ee5c577de91268 P cf512a43dd17478a9ef9e1c1c34d98a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf512a43dd17478a9ef9e1c1c34d98a1, State: Initialized, Role: FOLLOWER
I20260812 08:00:52.196512  6595 consensus_queue.cc:260] T ef4d86b0a9a64f1192ee5c577de91268 P 92d9c00c6a8c4679aaf96e4cb6586cbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88401325678c4312afd3853dd06928c1" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41357 } } peers { permanent_uuid: "cf512a43dd17478a9ef9e1c1c34d98a1" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 45103 } } peers { permanent_uuid: "92d9c00c6a8c4679aaf96e4cb6586cbc" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 44745 } }
I20260812 08:00:52.196812  6597 consensus_queue.cc:260] T ef4d86b0a9a64f1192ee5c577de91268 P cf512a43dd17478a9ef9e1c1c34d98a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88401325678c4312afd3853dd06928c1" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41357 } } peers { permanent_uuid: "cf512a43dd17478a9ef9e1c1c34d98a1" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 45103 } } peers { permanent_uuid: "92d9c00c6a8c4679aaf96e4cb6586cbc" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 44745 } }
I20260812 08:00:52.197060  6596 raft_consensus.cc:359] T ef4d86b0a9a64f1192ee5c577de91268 P 88401325678c4312afd3853dd06928c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88401325678c4312afd3853dd06928c1" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41357 } } peers { permanent_uuid: "cf512a43dd17478a9ef9e1c1c34d98a1" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 45103 } } peers { permanent_uuid: "92d9c00c6a8c4679aaf96e4cb6586cbc" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 44745 } }
I20260812 08:00:52.197270  6596 raft_consensus.cc:385] T ef4d86b0a9a64f1192ee5c577de91268 P 88401325678c4312afd3853dd06928c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:52.197331  6596 raft_consensus.cc:740] T ef4d86b0a9a64f1192ee5c577de91268 P 88401325678c4312afd3853dd06928c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88401325678c4312afd3853dd06928c1, State: Initialized, Role: FOLLOWER
I20260812 08:00:52.197340  6568 heartbeater.cc:499] Master 127.5.142.190:38049 was elected leader, sending a full tablet report...
I20260812 08:00:52.197535  6595 ts_tablet_manager.cc:1434] T ef4d86b0a9a64f1192ee5c577de91268 P 92d9c00c6a8c4679aaf96e4cb6586cbc: Time spent starting tablet: real 0.004s	user 0.003s	sys 0.001s
I20260812 08:00:52.197753  6596 consensus_queue.cc:260] T ef4d86b0a9a64f1192ee5c577de91268 P 88401325678c4312afd3853dd06928c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88401325678c4312afd3853dd06928c1" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41357 } } peers { permanent_uuid: "cf512a43dd17478a9ef9e1c1c34d98a1" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 45103 } } peers { permanent_uuid: "92d9c00c6a8c4679aaf96e4cb6586cbc" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 44745 } }
I20260812 08:00:52.197898  6597 ts_tablet_manager.cc:1434] T ef4d86b0a9a64f1192ee5c577de91268 P cf512a43dd17478a9ef9e1c1c34d98a1: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260812 08:00:52.198190  6327 heartbeater.cc:499] Master 127.5.142.190:38049 was elected leader, sending a full tablet report...
I20260812 08:00:52.198534  6596 ts_tablet_manager.cc:1434] T ef4d86b0a9a64f1192ee5c577de91268 P 88401325678c4312afd3853dd06928c1: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260812 08:00:52.198884  6098 heartbeater.cc:499] Master 127.5.142.190:38049 was elected leader, sending a full tablet report...
W20260812 08:00:52.258554  6100 tablet.cc:2406] T ef4d86b0a9a64f1192ee5c577de91268 P 88401325678c4312afd3853dd06928c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:00:52.372593  6571 tablet.cc:2406] T ef4d86b0a9a64f1192ee5c577de91268 P 92d9c00c6a8c4679aaf96e4cb6586cbc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:00:52.429464  6605 raft_consensus.cc:493] T ef4d86b0a9a64f1192ee5c577de91268 P 88401325678c4312afd3853dd06928c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:00:52.429701  6605 raft_consensus.cc:515] T ef4d86b0a9a64f1192ee5c577de91268 P 88401325678c4312afd3853dd06928c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88401325678c4312afd3853dd06928c1" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41357 } } peers { permanent_uuid: "cf512a43dd17478a9ef9e1c1c34d98a1" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 45103 } } peers { permanent_uuid: "92d9c00c6a8c4679aaf96e4cb6586cbc" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 44745 } }
I20260812 08:00:52.430930  6605 leader_election.cc:290] T ef4d86b0a9a64f1192ee5c577de91268 P 88401325678c4312afd3853dd06928c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf512a43dd17478a9ef9e1c1c34d98a1 (127.5.142.130:45103), 92d9c00c6a8c4679aaf96e4cb6586cbc (127.5.142.131:44745)
I20260812 08:00:52.435400  6475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4d86b0a9a64f1192ee5c577de91268" candidate_uuid: "88401325678c4312afd3853dd06928c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d9c00c6a8c4679aaf96e4cb6586cbc" is_pre_election: true
I20260812 08:00:52.435401  6250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4d86b0a9a64f1192ee5c577de91268" candidate_uuid: "88401325678c4312afd3853dd06928c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf512a43dd17478a9ef9e1c1c34d98a1" is_pre_election: true
I20260812 08:00:52.435693  6250 raft_consensus.cc:2468] T ef4d86b0a9a64f1192ee5c577de91268 P cf512a43dd17478a9ef9e1c1c34d98a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88401325678c4312afd3853dd06928c1 in term 0.
I20260812 08:00:52.435693  6475 raft_consensus.cc:2468] T ef4d86b0a9a64f1192ee5c577de91268 P 92d9c00c6a8c4679aaf96e4cb6586cbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88401325678c4312afd3853dd06928c1 in term 0.
I20260812 08:00:52.436224  5903 leader_election.cc:304] T ef4d86b0a9a64f1192ee5c577de91268 P 88401325678c4312afd3853dd06928c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88401325678c4312afd3853dd06928c1, cf512a43dd17478a9ef9e1c1c34d98a1; no voters: 
I20260812 08:00:52.436479  6605 raft_consensus.cc:2804] T ef4d86b0a9a64f1192ee5c577de91268 P 88401325678c4312afd3853dd06928c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:00:52.436554  6605 raft_consensus.cc:493] T ef4d86b0a9a64f1192ee5c577de91268 P 88401325678c4312afd3853dd06928c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:00:52.436601  6605 raft_consensus.cc:3060] T ef4d86b0a9a64f1192ee5c577de91268 P 88401325678c4312afd3853dd06928c1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:52.437629  6605 raft_consensus.cc:515] T ef4d86b0a9a64f1192ee5c577de91268 P 88401325678c4312afd3853dd06928c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88401325678c4312afd3853dd06928c1" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41357 } } peers { permanent_uuid: "cf512a43dd17478a9ef9e1c1c34d98a1" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 45103 } } peers { permanent_uuid: "92d9c00c6a8c4679aaf96e4cb6586cbc" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 44745 } }
I20260812 08:00:52.437984  6605 leader_election.cc:290] T ef4d86b0a9a64f1192ee5c577de91268 P 88401325678c4312afd3853dd06928c1 [CANDIDATE]: Term 1 election: Requested vote from peers cf512a43dd17478a9ef9e1c1c34d98a1 (127.5.142.130:45103), 92d9c00c6a8c4679aaf96e4cb6586cbc (127.5.142.131:44745)
I20260812 08:00:52.438385  6250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4d86b0a9a64f1192ee5c577de91268" candidate_uuid: "88401325678c4312afd3853dd06928c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf512a43dd17478a9ef9e1c1c34d98a1"
I20260812 08:00:52.438444  6475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef4d86b0a9a64f1192ee5c577de91268" candidate_uuid: "88401325678c4312afd3853dd06928c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d9c00c6a8c4679aaf96e4cb6586cbc"
I20260812 08:00:52.438493  6250 raft_consensus.cc:3060] T ef4d86b0a9a64f1192ee5c577de91268 P cf512a43dd17478a9ef9e1c1c34d98a1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:52.438572  6475 raft_consensus.cc:3060] T ef4d86b0a9a64f1192ee5c577de91268 P 92d9c00c6a8c4679aaf96e4cb6586cbc [term 0 FOLLOWER]: Advancing to term 1
W20260812 08:00:52.438720  6330 tablet.cc:2406] T ef4d86b0a9a64f1192ee5c577de91268 P cf512a43dd17478a9ef9e1c1c34d98a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:00:52.439505  6475 raft_consensus.cc:2468] T ef4d86b0a9a64f1192ee5c577de91268 P 92d9c00c6a8c4679aaf96e4cb6586cbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88401325678c4312afd3853dd06928c1 in term 1.
I20260812 08:00:52.439812  5903 leader_election.cc:304] T ef4d86b0a9a64f1192ee5c577de91268 P 88401325678c4312afd3853dd06928c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88401325678c4312afd3853dd06928c1, 92d9c00c6a8c4679aaf96e4cb6586cbc; no voters: 
I20260812 08:00:52.439977  6605 raft_consensus.cc:2804] T ef4d86b0a9a64f1192ee5c577de91268 P 88401325678c4312afd3853dd06928c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:00:52.439997  6250 raft_consensus.cc:2468] T ef4d86b0a9a64f1192ee5c577de91268 P cf512a43dd17478a9ef9e1c1c34d98a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88401325678c4312afd3853dd06928c1 in term 1.
I20260812 08:00:52.440192  6605 raft_consensus.cc:697] T ef4d86b0a9a64f1192ee5c577de91268 P 88401325678c4312afd3853dd06928c1 [term 1 LEADER]: Becoming Leader. State: Replica: 88401325678c4312afd3853dd06928c1, State: Running, Role: LEADER
I20260812 08:00:52.440510  6605 consensus_queue.cc:237] T ef4d86b0a9a64f1192ee5c577de91268 P 88401325678c4312afd3853dd06928c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88401325678c4312afd3853dd06928c1" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41357 } } peers { permanent_uuid: "cf512a43dd17478a9ef9e1c1c34d98a1" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 45103 } } peers { permanent_uuid: "92d9c00c6a8c4679aaf96e4cb6586cbc" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 44745 } }
I20260812 08:00:52.443661  5750 catalog_manager.cc:5719] T ef4d86b0a9a64f1192ee5c577de91268 P 88401325678c4312afd3853dd06928c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88401325678c4312afd3853dd06928c1 (127.5.142.129). New cstate: current_term: 1 leader_uuid: "88401325678c4312afd3853dd06928c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88401325678c4312afd3853dd06928c1" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf512a43dd17478a9ef9e1c1c34d98a1" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 45103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92d9c00c6a8c4679aaf96e4cb6586cbc" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 44745 } health_report { overall_health: UNKNOWN } } }
I20260812 08:00:52.468272  5690 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:00:52.470227  5690 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 88401325678c4312afd3853dd06928c1 to finish bootstrapping
I20260812 08:00:52.475953  5690 ts_itest-base.cc:246] Waiting for 1 tablets on tserver cf512a43dd17478a9ef9e1c1c34d98a1 to finish bootstrapping
I20260812 08:00:52.480990  5690 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 92d9c00c6a8c4679aaf96e4cb6586cbc to finish bootstrapping
I20260812 08:00:52.498220  5690 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 0.0 1.1
I20260812 08:00:52.601330  5690 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 0.0 1.1
I20260812 08:00:52.804718  5690 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 0.0 1.1
I20260812 08:00:52.925390  6605 consensus_queue.cc:1048] T ef4d86b0a9a64f1192ee5c577de91268 P 88401325678c4312afd3853dd06928c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf512a43dd17478a9ef9e1c1c34d98a1" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 45103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:00:52.930958  6605 consensus_queue.cc:1048] T ef4d86b0a9a64f1192ee5c577de91268 P 88401325678c4312afd3853dd06928c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92d9c00c6a8c4679aaf96e4cb6586cbc" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 44745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 08:00:53.114132  6617 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:51026: Not implemented: Unknown operation type: 115
I20260812 08:00:53.195351  6436 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 08:00:53.195356  5982 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260812 08:00:53.205687  6212 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |       Address       | Status
----------------------------------+---------------------+---------
 28e2d3640dd24347b929e312c6440afa | 127.5.142.190:38049 | HEALTHY

Unusual flags for Master:
               Flag               |                                                                                Value                                                                                 |      Tags       |         Master
----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
 ipki_ca_key_size                 | 768                                                                                                                                                                  | experimental    | all 1 server(s) checked
 ipki_server_key_size             | 768                                                                                                                                                                  | experimental    | all 1 server(s) checked
 never_fsync                      | true                                                                                                                                                                 | unsafe,advanced | all 1 server(s) checked
 openssl_security_level_override  | 0                                                                                                                                                                    | unsafe,hidden   | all 1 server(s) checked
 rpc_reuseport                    | true                                                                                                                                                                 | experimental    | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports | true                                                                                                                                                                 | unsafe          | all 1 server(s) checked
 server_dump_info_format          | pb                                                                                                                                                                   | hidden          | all 1 server(s) checked
 server_dump_info_path            | /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden          | all 1 server(s) checked
 tsk_num_rsa_bits                 | 512                                                                                                                                                                  | experimental    | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |        Value        |         Master
---------------------+---------------------+-------------------------
 builtin_ntp_servers | 127.5.142.148:42237 | all 1 server(s) checked
 time_source         | builtin             | all 1 server(s) checked

Tablet Server Summary
               UUID               |       Address       | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
 88401325678c4312afd3853dd06928c1 | 127.5.142.129:41357 | HEALTHY | <none>   |       1        |       0
 92d9c00c6a8c4679aaf96e4cb6586cbc | 127.5.142.131:44745 | HEALTHY | <none>   |       0        |       0
 cf512a43dd17478a9ef9e1c1c34d98a1 | 127.5.142.130:45103 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
               Flag               |                                                                              Value                                                                               |      Tags       |      Tablet Server
----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
 ipki_server_key_size             | 768                                                                                                                                                              | experimental    | all 3 server(s) checked
 local_ip_for_outbound_sockets    | 127.5.142.129                                                                                                                                                    | experimental    | 127.5.142.129:41357
 local_ip_for_outbound_sockets    | 127.5.142.130                                                                                                                                                    | experimental    | 127.5.142.130:45103
 local_ip_for_outbound_sockets    | 127.5.142.131                                                                                                                                                    | experimental    | 127.5.142.131:44745
 never_fsync                      | true                                                                                                                                                             | unsafe,advanced | all 3 server(s) checked
 openssl_security_level_override  | 0                                                                                                                                                                | unsafe,hidden   | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports | true                                                                                                                                                             | unsafe          | all 3 server(s) checked
 server_dump_info_format          | pb                                                                                                                                                               | hidden          | all 3 server(s) checked
 server_dump_info_path            | /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden          | 127.5.142.129:41357
 server_dump_info_path            | /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden          | 127.5.142.130:45103
 server_dump_info_path            | /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden          | 127.5.142.131:44745

Flags of checked categories for Tablet Server:
        Flag         |        Value        |      Tablet Server
---------------------+---------------------+-------------------------
 builtin_ntp_servers | 127.5.142.148:42237 | all 3 server(s) checked
 time_source         | builtin             | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
   Name    | RF | Status  | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-----------+----+---------+---------------+---------+------------+------------------+-------------
 TestTable | 3  | HEALTHY | 1             | 1       | 0          | 0                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

OK
I20260812 08:00:53.231161  5690 log_verifier.cc:126] Checking tablet ef4d86b0a9a64f1192ee5c577de91268
I20260812 08:00:53.257087  5690 log_verifier.cc:177] Verified matching terms for 2 ops in tablet ef4d86b0a9a64f1192ee5c577de91268
I20260812 08:00:53.644027  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 5876
I20260812 08:00:53.661231  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 6102
I20260812 08:00:53.677874  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 6337
I20260812 08:00:53.694468  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 5705
2026-08-12T08:00:53Z chronyd exiting
[       OK ] RaftConsensusITest.TestFailedOp (2452 ms)
[ RUN      ] RaftConsensusITest.MultiThreadedInsertWithFailovers
I20260812 08:00:53.729570  5690 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:00:53.729625  5690 ts_itest-base.cc:116] --------------
I20260812 08:00:53.729641  5690 ts_itest-base.cc:117] 7 tablet servers
I20260812 08:00:53.729655  5690 ts_itest-base.cc:118] 7 replicas per TS
I20260812 08:00:53.729667  5690 ts_itest-base.cc:119] --------------
2026-08-12T08:00:53Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:00:53Z Disabled control of system clock
I20260812 08:00:53.750005  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.142.190:44379
--webserver_interface=127.5.142.190
--webserver_port=0
--builtin_ntp_servers=127.5.142.148:43055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.142.190:44379 with env {}
W20260812 08:00:53.858631  6703 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:53.858870  6703 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:53.858912  6703 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:53.862354  6703 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:00:53.862422  6703 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:53.862442  6703 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:00:53.862458  6703 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:00:53.866488  6703 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:43055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.142.190:44379
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.142.190:44379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.5.142.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.6703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:53.867645  6703 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:53.868800  6703 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:53.874547  6724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:53.874534  6722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:53.874534  6720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:53.874897  6703 server_base.cc:1061] running on GCE node
I20260812 08:00:53.875478  6703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:53.876336  6703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:53.877530  6703 hybrid_clock.cc:648] HybridClock initialized: now 1786521653877509 us; error 53 us; skew 500 ppm
I20260812 08:00:53.879384  6703 webserver.cc:533] Webserver started at http://127.5.142.190:38317/ using document root <none> and password file <none>
I20260812 08:00:53.880129  6703 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:53.880203  6703 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:53.880411  6703 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:53.882017  6703 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "23389c6fd7634cbca868c16d3f1c61d2"
format_stamp: "Formatted at 2026-08-12 08:00:53 on dist-test-slave-ncp9"
I20260812 08:00:53.882504  6703 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "23389c6fd7634cbca868c16d3f1c61d2"
format_stamp: "Formatted at 2026-08-12 08:00:53 on dist-test-slave-ncp9"
I20260812 08:00:53.885869  6703 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.001s
I20260812 08:00:53.888180  6731 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:53.889359  6703 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:00:53.889472  6703 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
uuid: "23389c6fd7634cbca868c16d3f1c61d2"
format_stamp: "Formatted at 2026-08-12 08:00:53 on dist-test-slave-ncp9"
I20260812 08:00:53.889561  6703 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:53.933861  6703 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:53.934569  6703 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:53.934747  6703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:53.942389  6703 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.190:44379
I20260812 08:00:53.942399  6804 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.190:44379 every 8 connection(s)
I20260812 08:00:53.943403  6703 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:00:53.946265  6805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:53.946538  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 6703
I20260812 08:00:53.946657  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:00:53.953368  6805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23389c6fd7634cbca868c16d3f1c61d2: Bootstrap starting.
I20260812 08:00:53.955770  6805 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23389c6fd7634cbca868c16d3f1c61d2: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:53.956632  6805 log.cc:826] T 00000000000000000000000000000000 P 23389c6fd7634cbca868c16d3f1c61d2: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:53.958372  6805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23389c6fd7634cbca868c16d3f1c61d2: No bootstrap required, opened a new log
I20260812 08:00:53.960834  6805 raft_consensus.cc:359] T 00000000000000000000000000000000 P 23389c6fd7634cbca868c16d3f1c61d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23389c6fd7634cbca868c16d3f1c61d2" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 44379 } }
I20260812 08:00:53.961074  6805 raft_consensus.cc:385] T 00000000000000000000000000000000 P 23389c6fd7634cbca868c16d3f1c61d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:53.961140  6805 raft_consensus.cc:740] T 00000000000000000000000000000000 P 23389c6fd7634cbca868c16d3f1c61d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23389c6fd7634cbca868c16d3f1c61d2, State: Initialized, Role: FOLLOWER
I20260812 08:00:53.961539  6805 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23389c6fd7634cbca868c16d3f1c61d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23389c6fd7634cbca868c16d3f1c61d2" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 44379 } }
I20260812 08:00:53.961674  6805 raft_consensus.cc:399] T 00000000000000000000000000000000 P 23389c6fd7634cbca868c16d3f1c61d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:00:53.961738  6805 raft_consensus.cc:493] T 00000000000000000000000000000000 P 23389c6fd7634cbca868c16d3f1c61d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:00:53.961845  6805 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 23389c6fd7634cbca868c16d3f1c61d2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:53.962720  6805 raft_consensus.cc:515] T 00000000000000000000000000000000 P 23389c6fd7634cbca868c16d3f1c61d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23389c6fd7634cbca868c16d3f1c61d2" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 44379 } }
I20260812 08:00:53.963068  6805 leader_election.cc:304] T 00000000000000000000000000000000 P 23389c6fd7634cbca868c16d3f1c61d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23389c6fd7634cbca868c16d3f1c61d2; no voters: 
I20260812 08:00:53.963356  6805 leader_election.cc:290] T 00000000000000000000000000000000 P 23389c6fd7634cbca868c16d3f1c61d2 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:00:53.963487  6814 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 23389c6fd7634cbca868c16d3f1c61d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:00:53.963680  6814 raft_consensus.cc:697] T 00000000000000000000000000000000 P 23389c6fd7634cbca868c16d3f1c61d2 [term 1 LEADER]: Becoming Leader. State: Replica: 23389c6fd7634cbca868c16d3f1c61d2, State: Running, Role: LEADER
I20260812 08:00:53.963996  6814 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23389c6fd7634cbca868c16d3f1c61d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23389c6fd7634cbca868c16d3f1c61d2" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 44379 } }
I20260812 08:00:53.964459  6805 sys_catalog.cc:565] T 00000000000000000000000000000000 P 23389c6fd7634cbca868c16d3f1c61d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:00:53.965932  6816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23389c6fd7634cbca868c16d3f1c61d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23389c6fd7634cbca868c16d3f1c61d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23389c6fd7634cbca868c16d3f1c61d2" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 44379 } } }
I20260812 08:00:53.965987  6817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23389c6fd7634cbca868c16d3f1c61d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23389c6fd7634cbca868c16d3f1c61d2. Latest consensus state: current_term: 1 leader_uuid: "23389c6fd7634cbca868c16d3f1c61d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23389c6fd7634cbca868c16d3f1c61d2" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 44379 } } }
I20260812 08:00:53.966058  6816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23389c6fd7634cbca868c16d3f1c61d2 [sys.catalog]: This master's current role is: LEADER
I20260812 08:00:53.966077  6817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23389c6fd7634cbca868c16d3f1c61d2 [sys.catalog]: This master's current role is: LEADER
I20260812 08:00:53.967134  6830 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:00:53.969779  6830 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:00:53.975184  6830 catalog_manager.cc:1383] Generated new cluster ID: 312677b9cb31410dbf8e1dcd6f792c9c
I20260812 08:00:53.975242  6830 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:00:53.987005  6830 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:00:53.987782  6830 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:00:53.998027  6830 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 23389c6fd7634cbca868c16d3f1c61d2: Generated new TSK 0
I20260812 08:00:53.998742  6830 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:00:54.009572  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.129:0
--local_ip_for_outbound_sockets=127.5.142.129
--webserver_interface=127.5.142.129
--webserver_port=0
--tserver_master_addrs=127.5.142.190:44379
--builtin_ntp_servers=127.5.142.148:43055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
W20260812 08:00:54.142827  6842 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:54.143087  6842 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:54.143149  6842 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:54.146678  6842 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:54.146839  6842 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.129
I20260812 08:00:54.151001  6842 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:43055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.5.142.129
--webserver_port=0
--tserver_master_addrs=127.5.142.190:44379
--never_fsync=true
--heap_profile_path=/tmp/kudu.6842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.129

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:54.152209  6842 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:54.153494  6842 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:54.160282  6849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:54.160283  6850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:54.160282  6853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:54.160393  6842 server_base.cc:1061] running on GCE node
I20260812 08:00:54.160981  6842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:54.161546  6842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:54.162707  6842 hybrid_clock.cc:648] HybridClock initialized: now 1786521654162694 us; error 30 us; skew 500 ppm
I20260812 08:00:54.164463  6842 webserver.cc:533] Webserver started at http://127.5.142.129:33453/ using document root <none> and password file <none>
I20260812 08:00:54.165091  6842 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:54.165153  6842 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:54.165350  6842 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:54.166900  6842 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "3ae3f7a6e9f943e996a6dcea504878d0"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-ncp9"
I20260812 08:00:54.167477  6842 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "3ae3f7a6e9f943e996a6dcea504878d0"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-ncp9"
I20260812 08:00:54.170691  6842 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:00:54.172809  6866 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:54.173758  6842 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:00:54.173863  6842 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "3ae3f7a6e9f943e996a6dcea504878d0"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-ncp9"
I20260812 08:00:54.173943  6842 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:54.196398  6842 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:54.197165  6842 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:54.197346  6842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:54.197964  6842 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:54.199018  6842 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:54.199069  6842 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:54.199121  6842 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:54.199149  6842 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:54.208149  6842 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.129:41863
I20260812 08:00:54.208213  7036 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.129:41863 every 8 connection(s)
I20260812 08:00:54.209156  6842 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:00:54.217284  7039 heartbeater.cc:344] Connected to a master server at 127.5.142.190:44379
I20260812 08:00:54.217525  7039 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:54.218035  7039 heartbeater.cc:507] Master 127.5.142.190:44379 requested a full tablet report, sending...
I20260812 08:00:54.218266  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 6842
I20260812 08:00:54.218364  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:00:54.219692  6753 ts_manager.cc:194] Registered new tserver with Master: 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863)
I20260812 08:00:54.220621  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.130:0
--local_ip_for_outbound_sockets=127.5.142.130
--webserver_interface=127.5.142.130
--webserver_port=0
--tserver_master_addrs=127.5.142.190:44379
--builtin_ntp_servers=127.5.142.148:43055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
I20260812 08:00:54.220995  6753 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.129:44731
W20260812 08:00:54.327991  7043 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:54.328218  7043 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:54.328261  7043 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:54.331624  7043 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:54.331735  7043 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.130
I20260812 08:00:54.335500  7043 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:43055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.5.142.130
--webserver_port=0
--tserver_master_addrs=127.5.142.190:44379
--never_fsync=true
--heap_profile_path=/tmp/kudu.7043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.130

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:54.336591  7043 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:54.337756  7043 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:54.344086  7059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:54.344086  7056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:54.344146  7057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:54.344309  7043 server_base.cc:1061] running on GCE node
I20260812 08:00:54.344673  7043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:54.345239  7043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:54.346401  7043 hybrid_clock.cc:648] HybridClock initialized: now 1786521654346382 us; error 34 us; skew 500 ppm
I20260812 08:00:54.348243  7043 webserver.cc:533] Webserver started at http://127.5.142.130:38317/ using document root <none> and password file <none>
I20260812 08:00:54.348824  7043 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:54.348920  7043 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:54.349143  7043 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:54.350735  7043 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "56b60424ed9f4a079689ad82a318d29a"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-ncp9"
I20260812 08:00:54.351236  7043 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "56b60424ed9f4a079689ad82a318d29a"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-ncp9"
I20260812 08:00:54.354779  7043 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.002s
I20260812 08:00:54.356940  7073 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:54.357972  7043 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:00:54.358084  7043 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "56b60424ed9f4a079689ad82a318d29a"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-ncp9"
I20260812 08:00:54.358173  7043 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:54.375376  7043 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:54.376048  7043 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:54.376221  7043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:54.376811  7043 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:54.377946  7043 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:54.377996  7043 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:54.378052  7043 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:54.378077  7043 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:54.387024  7043 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.130:38363
I20260812 08:00:54.387089  7253 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.130:38363 every 8 connection(s)
I20260812 08:00:54.387984  7043 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:00:54.398499  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 7043
I20260812 08:00:54.398626  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:00:54.400436  7254 heartbeater.cc:344] Connected to a master server at 127.5.142.190:44379
I20260812 08:00:54.400705  7254 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:54.401094  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.131:0
--local_ip_for_outbound_sockets=127.5.142.131
--webserver_interface=127.5.142.131
--webserver_port=0
--tserver_master_addrs=127.5.142.190:44379
--builtin_ntp_servers=127.5.142.148:43055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
I20260812 08:00:54.401342  7254 heartbeater.cc:507] Master 127.5.142.190:44379 requested a full tablet report, sending...
I20260812 08:00:54.402437  6753 ts_manager.cc:194] Registered new tserver with Master: 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363)
I20260812 08:00:54.402994  6753 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.130:34267
W20260812 08:00:54.509900  7258 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:54.510114  7258 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:54.510183  7258 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:54.513469  7258 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:54.513582  7258 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.131
I20260812 08:00:54.517325  7258 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:43055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.5.142.131
--webserver_port=0
--tserver_master_addrs=127.5.142.190:44379
--never_fsync=true
--heap_profile_path=/tmp/kudu.7258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.131

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:54.518385  7258 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:54.519537  7258 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:54.526137  7268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:54.526127  7270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:54.526124  7273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:54.526328  7258 server_base.cc:1061] running on GCE node
I20260812 08:00:54.526767  7258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:54.527305  7258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:54.528456  7258 hybrid_clock.cc:648] HybridClock initialized: now 1786521654528445 us; error 27 us; skew 500 ppm
I20260812 08:00:54.530191  7258 webserver.cc:533] Webserver started at http://127.5.142.131:43149/ using document root <none> and password file <none>
I20260812 08:00:54.530743  7258 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:54.530802  7258 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:54.530980  7258 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:54.532519  7258 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "16901c5559a748a9aa770d73e254099c"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-ncp9"
I20260812 08:00:54.533036  7258 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "16901c5559a748a9aa770d73e254099c"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-ncp9"
I20260812 08:00:54.536301  7258 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:00:54.538589  7282 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:54.539573  7258 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.001s
I20260812 08:00:54.539681  7258 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "16901c5559a748a9aa770d73e254099c"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-ncp9"
I20260812 08:00:54.539770  7258 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:54.553190  7258 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:54.553843  7258 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:54.554015  7258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:54.554585  7258 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:54.555724  7258 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:54.555775  7258 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:54.555830  7258 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:54.555855  7258 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:54.566370  7258 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.131:33183
I20260812 08:00:54.566416  7459 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.131:33183 every 8 connection(s)
I20260812 08:00:54.567341  7258 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:00:54.569105  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 7258
I20260812 08:00:54.569211  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:00:54.571730  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.132:0
--local_ip_for_outbound_sockets=127.5.142.132
--webserver_interface=127.5.142.132
--webserver_port=0
--tserver_master_addrs=127.5.142.190:44379
--builtin_ntp_servers=127.5.142.148:43055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
I20260812 08:00:54.576098  7461 heartbeater.cc:344] Connected to a master server at 127.5.142.190:44379
I20260812 08:00:54.576332  7461 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:54.576839  7461 heartbeater.cc:507] Master 127.5.142.190:44379 requested a full tablet report, sending...
I20260812 08:00:54.577943  6753 ts_manager.cc:194] Registered new tserver with Master: 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183)
I20260812 08:00:54.578742  6753 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.131:35767
W20260812 08:00:54.683837  7466 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:54.684078  7466 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:54.684122  7466 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:54.687500  7466 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:54.687616  7466 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.132
I20260812 08:00:54.691483  7466 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:43055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.5.142.132
--webserver_port=0
--tserver_master_addrs=127.5.142.190:44379
--never_fsync=true
--heap_profile_path=/tmp/kudu.7466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.132

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:54.692639  7466 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:54.693854  7466 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:54.700373  7477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:54.700366  7479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:54.700318  7481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:54.700707  7466 server_base.cc:1061] running on GCE node
I20260812 08:00:54.701088  7466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:54.701623  7466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:54.702790  7466 hybrid_clock.cc:648] HybridClock initialized: now 1786521654702771 us; error 28 us; skew 500 ppm
I20260812 08:00:54.704407  7466 webserver.cc:533] Webserver started at http://127.5.142.132:43191/ using document root <none> and password file <none>
I20260812 08:00:54.704985  7466 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:54.705050  7466 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:54.705260  7466 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:54.706718  7466 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "faafbc72cbd444ff90977a2cdb091b72"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-ncp9"
I20260812 08:00:54.707175  7466 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "faafbc72cbd444ff90977a2cdb091b72"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-ncp9"
I20260812 08:00:54.710345  7466 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:00:54.712396  7491 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:54.713460  7466 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:00:54.713572  7466 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "faafbc72cbd444ff90977a2cdb091b72"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-ncp9"
I20260812 08:00:54.713660  7466 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:54.735958  7466 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:54.736668  7466 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:54.736852  7466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:54.737504  7466 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:54.738610  7466 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:54.738670  7466 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:54.738726  7466 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:54.738754  7466 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:54.750164  7466 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.132:43153
I20260812 08:00:54.750185  7683 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.132:43153 every 8 connection(s)
I20260812 08:00:54.751125  7466 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20260812 08:00:54.758909  7684 heartbeater.cc:344] Connected to a master server at 127.5.142.190:44379
I20260812 08:00:54.759145  7684 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:54.759654  7684 heartbeater.cc:507] Master 127.5.142.190:44379 requested a full tablet report, sending...
I20260812 08:00:54.759758  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 7466
I20260812 08:00:54.759867  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20260812 08:00:54.760774  6753 ts_manager.cc:194] Registered new tserver with Master: faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153)
I20260812 08:00:54.761369  6753 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.132:35099
I20260812 08:00:54.762005  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.133:0
--local_ip_for_outbound_sockets=127.5.142.133
--webserver_interface=127.5.142.133
--webserver_port=0
--tserver_master_addrs=127.5.142.190:44379
--builtin_ntp_servers=127.5.142.148:43055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
W20260812 08:00:54.869736  7687 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:54.869948  7687 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:54.870004  7687 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:54.873303  7687 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:54.873418  7687 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.133
I20260812 08:00:54.877300  7687 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:43055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.133:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.5.142.133
--webserver_port=0
--tserver_master_addrs=127.5.142.190:44379
--never_fsync=true
--heap_profile_path=/tmp/kudu.7687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.133

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:54.878436  7687 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:54.879657  7687 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:54.886298  7701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:54.886270  7699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:54.886332  7696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:54.886729  7687 server_base.cc:1061] running on GCE node
I20260812 08:00:54.887113  7687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:54.887663  7687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:54.888814  7687 hybrid_clock.cc:648] HybridClock initialized: now 1786521654888799 us; error 32 us; skew 500 ppm
I20260812 08:00:54.890589  7687 webserver.cc:533] Webserver started at http://127.5.142.133:36547/ using document root <none> and password file <none>
I20260812 08:00:54.891161  7687 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:54.891216  7687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:54.891407  7687 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:54.893034  7687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "1664e32d3cd0473592085aa522379f22"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-ncp9"
I20260812 08:00:54.893605  7687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "1664e32d3cd0473592085aa522379f22"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-ncp9"
I20260812 08:00:54.896965  7687 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:00:54.899286  7711 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:54.900309  7687 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:00:54.900411  7687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "1664e32d3cd0473592085aa522379f22"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-ncp9"
I20260812 08:00:54.900506  7687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:54.910760  7687 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:54.911453  7687 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:54.911626  7687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:54.912189  7687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:54.913358  7687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:54.913405  7687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:54.913445  7687 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:54.913473  7687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:54.925127  7687 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.133:33105
I20260812 08:00:54.925173  7873 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.133:33105 every 8 connection(s)
I20260812 08:00:54.926072  7687 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20260812 08:00:54.929303  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 7687
I20260812 08:00:54.929430  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20260812 08:00:54.931708  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.134:0
--local_ip_for_outbound_sockets=127.5.142.134
--webserver_interface=127.5.142.134
--webserver_port=0
--tserver_master_addrs=127.5.142.190:44379
--builtin_ntp_servers=127.5.142.148:43055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
I20260812 08:00:54.935221  7874 heartbeater.cc:344] Connected to a master server at 127.5.142.190:44379
I20260812 08:00:54.935461  7874 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:54.935984  7874 heartbeater.cc:507] Master 127.5.142.190:44379 requested a full tablet report, sending...
I20260812 08:00:54.937124  6753 ts_manager.cc:194] Registered new tserver with Master: 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105)
I20260812 08:00:54.937847  6753 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.133:37971
W20260812 08:00:55.041131  7878 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:55.041352  7878 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:55.041399  7878 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:55.044672  7878 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:55.044775  7878 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.134
I20260812 08:00:55.048550  7878 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:43055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.134:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.5.142.134
--webserver_port=0
--tserver_master_addrs=127.5.142.190:44379
--never_fsync=true
--heap_profile_path=/tmp/kudu.7878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.134

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:55.049695  7878 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:55.050835  7878 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:55.057451  7885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:55.057421  7890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:55.057423  7886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:55.057729  7878 server_base.cc:1061] running on GCE node
I20260812 08:00:55.058085  7878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:55.058652  7878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:55.059813  7878 hybrid_clock.cc:648] HybridClock initialized: now 1786521655059790 us; error 34 us; skew 500 ppm
I20260812 08:00:55.061518  7878 webserver.cc:533] Webserver started at http://127.5.142.134:34013/ using document root <none> and password file <none>
I20260812 08:00:55.062079  7878 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:55.062139  7878 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:55.062338  7878 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:55.063866  7878 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "780d7e64a8214418a371f35351b113df"
format_stamp: "Formatted at 2026-08-12 08:00:55 on dist-test-slave-ncp9"
I20260812 08:00:55.064318  7878 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "780d7e64a8214418a371f35351b113df"
format_stamp: "Formatted at 2026-08-12 08:00:55 on dist-test-slave-ncp9"
I20260812 08:00:55.067575  7878 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:00:55.069674  7897 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:55.070590  7878 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:00:55.070701  7878 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "780d7e64a8214418a371f35351b113df"
format_stamp: "Formatted at 2026-08-12 08:00:55 on dist-test-slave-ncp9"
I20260812 08:00:55.070784  7878 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:55.086373  7878 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:55.087122  7878 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:55.087322  7878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:55.087971  7878 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:55.089196  7878 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:55.089255  7878 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:55.089324  7878 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:55.089356  7878 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:55.101665  7878 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.134:33021
I20260812 08:00:55.101691  8085 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.134:33021 every 8 connection(s)
I20260812 08:00:55.102742  7878 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20260812 08:00:55.111238  8086 heartbeater.cc:344] Connected to a master server at 127.5.142.190:44379
I20260812 08:00:55.111485  8086 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:55.111929  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 7878
I20260812 08:00:55.112042  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20260812 08:00:55.112071  8086 heartbeater.cc:507] Master 127.5.142.190:44379 requested a full tablet report, sending...
I20260812 08:00:55.113234  6751 ts_manager.cc:194] Registered new tserver with Master: 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:00:55.113847  6751 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.134:57823
I20260812 08:00:55.114494  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.135:0
--local_ip_for_outbound_sockets=127.5.142.135
--webserver_interface=127.5.142.135
--webserver_port=0
--tserver_master_addrs=127.5.142.190:44379
--builtin_ntp_servers=127.5.142.148:43055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
I20260812 08:00:55.223181  7039 heartbeater.cc:499] Master 127.5.142.190:44379 was elected leader, sending a full tablet report...
W20260812 08:00:55.230204  8090 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:00:55.230438  8090 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:00:55.230481  8090 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:00:55.234014  8090 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:00:55.234129  8090 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.135
I20260812 08:00:55.238073  8090 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:43055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.135:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.5.142.135
--webserver_port=0
--tserver_master_addrs=127.5.142.190:44379
--never_fsync=true
--heap_profile_path=/tmp/kudu.8090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.135

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:00:55.239269  8090 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:00:55.240470  8090 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:00:55.247103  8099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:55.247165  8101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:00:55.247109  8098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:00:55.247545  8090 server_base.cc:1061] running on GCE node
I20260812 08:00:55.247941  8090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:00:55.248509  8090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:00:55.249688  8090 hybrid_clock.cc:648] HybridClock initialized: now 1786521655249674 us; error 25 us; skew 500 ppm
I20260812 08:00:55.251559  8090 webserver.cc:533] Webserver started at http://127.5.142.135:33815/ using document root <none> and password file <none>
I20260812 08:00:55.252233  8090 fs_manager.cc:362] Metadata directory not provided
I20260812 08:00:55.252296  8090 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:00:55.252496  8090 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:00:55.254222  8090 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "6801d45ca6614156906e5d571ebcb8a6"
format_stamp: "Formatted at 2026-08-12 08:00:55 on dist-test-slave-ncp9"
I20260812 08:00:55.254714  8090 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "6801d45ca6614156906e5d571ebcb8a6"
format_stamp: "Formatted at 2026-08-12 08:00:55 on dist-test-slave-ncp9"
I20260812 08:00:55.258307  8090 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.002s
I20260812 08:00:55.260419  8111 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:55.261487  8090 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:00:55.261616  8090 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "6801d45ca6614156906e5d571ebcb8a6"
format_stamp: "Formatted at 2026-08-12 08:00:55 on dist-test-slave-ncp9"
I20260812 08:00:55.261708  8090 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:00:55.285215  8090 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:00:55.285995  8090 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:00:55.286175  8090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:00:55.286796  8090 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:00:55.288007  8090 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:00:55.288057  8090 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:55.288113  8090 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:00:55.288139  8090 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:00:55.298964  8090 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.135:42409
I20260812 08:00:55.298987  8299 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.135:42409 every 8 connection(s)
I20260812 08:00:55.299918  8090 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20260812 08:00:55.303018  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 8090
I20260812 08:00:55.303153  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20260812 08:00:55.308262  8301 heartbeater.cc:344] Connected to a master server at 127.5.142.190:44379
I20260812 08:00:55.308493  8301 heartbeater.cc:461] Registering TS with master...
I20260812 08:00:55.309034  8301 heartbeater.cc:507] Master 127.5.142.190:44379 requested a full tablet report, sending...
I20260812 08:00:55.309980  6751 ts_manager.cc:194] Registered new tserver with Master: 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409)
I20260812 08:00:55.310523  6751 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.135:48589
I20260812 08:00:55.317489  5690 external_mini_cluster.cc:955] 7 TS(s) registered with all masters
I20260812 08:00:55.326851  6751 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38922:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
W20260812 08:00:55.329416  6751 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20260812 08:00:55.350061  6936 tablet_service.cc:1511] Processing CreateTablet for tablet f22cf4f6412248aa9e68c6d5d61153e8 (DEFAULT_TABLE table=TestTable [id=c9524fde058f45f0b30e492880ff3571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:55.350554  7774 tablet_service.cc:1511] Processing CreateTablet for tablet f22cf4f6412248aa9e68c6d5d61153e8 (DEFAULT_TABLE table=TestTable [id=c9524fde058f45f0b30e492880ff3571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:55.351258  6936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f22cf4f6412248aa9e68c6d5d61153e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:55.351619  7774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f22cf4f6412248aa9e68c6d5d61153e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:55.356050  7153 tablet_service.cc:1511] Processing CreateTablet for tablet f22cf4f6412248aa9e68c6d5d61153e8 (DEFAULT_TABLE table=TestTable [id=c9524fde058f45f0b30e492880ff3571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:55.357070  7153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f22cf4f6412248aa9e68c6d5d61153e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:55.357086  8192 tablet_service.cc:1511] Processing CreateTablet for tablet f22cf4f6412248aa9e68c6d5d61153e8 (DEFAULT_TABLE table=TestTable [id=c9524fde058f45f0b30e492880ff3571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:55.358064  8192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f22cf4f6412248aa9e68c6d5d61153e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:55.363987  8343 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Bootstrap starting.
I20260812 08:00:55.363965  7562 tablet_service.cc:1511] Processing CreateTablet for tablet f22cf4f6412248aa9e68c6d5d61153e8 (DEFAULT_TABLE table=TestTable [id=c9524fde058f45f0b30e492880ff3571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:55.365048  7562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f22cf4f6412248aa9e68c6d5d61153e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:55.366322  8343 tablet_bootstrap.cc:654] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:55.367216  8343 log.cc:826] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:55.367533  8344 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6: Bootstrap starting.
I20260812 08:00:55.369139  8343 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: No bootstrap required, opened a new log
I20260812 08:00:55.369335  8343 ts_tablet_manager.cc:1403] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Time spent bootstrapping tablet: real 0.006s	user 0.005s	sys 0.000s
I20260812 08:00:55.369910  8344 tablet_bootstrap.cc:654] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:55.370323  7356 tablet_service.cc:1511] Processing CreateTablet for tablet f22cf4f6412248aa9e68c6d5d61153e8 (DEFAULT_TABLE table=TestTable [id=c9524fde058f45f0b30e492880ff3571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:55.370738  8344 log.cc:826] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:55.371260  7356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f22cf4f6412248aa9e68c6d5d61153e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:55.371306  8347 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Bootstrap starting.
I20260812 08:00:55.371506  8342 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Bootstrap starting.
I20260812 08:00:55.372309  8343 raft_consensus.cc:359] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:00:55.372534  8343 raft_consensus.cc:385] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:55.372593  8343 raft_consensus.cc:740] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56b60424ed9f4a079689ad82a318d29a, State: Initialized, Role: FOLLOWER
I20260812 08:00:55.373191  8343 consensus_queue.cc:260] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:00:55.373322  8347 tablet_bootstrap.cc:654] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:55.373672  8342 tablet_bootstrap.cc:654] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:55.373780  8341 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Bootstrap starting.
I20260812 08:00:55.374228  8343 ts_tablet_manager.cc:1434] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:00:55.374571  8342 log.cc:826] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:55.374619  7254 heartbeater.cc:499] Master 127.5.142.190:44379 was elected leader, sending a full tablet report...
I20260812 08:00:55.375591  8347 log.cc:826] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:55.378299  8344 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6: No bootstrap required, opened a new log
I20260812 08:00:55.378608  8344 ts_tablet_manager.cc:1403] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6: Time spent bootstrapping tablet: real 0.011s	user 0.005s	sys 0.000s
I20260812 08:00:55.378724  7975 tablet_service.cc:1511] Processing CreateTablet for tablet f22cf4f6412248aa9e68c6d5d61153e8 (DEFAULT_TABLE table=TestTable [id=c9524fde058f45f0b30e492880ff3571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:00:55.379715  7975 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f22cf4f6412248aa9e68c6d5d61153e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:00:55.379747  8341 tablet_bootstrap.cc:654] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:55.380501  8341 log.cc:826] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:55.381150  8352 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c: Bootstrap starting.
I20260812 08:00:55.381770  8344 raft_consensus.cc:359] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:00:55.382032  8344 raft_consensus.cc:385] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:55.382104  8344 raft_consensus.cc:740] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6801d45ca6614156906e5d571ebcb8a6, State: Initialized, Role: FOLLOWER
I20260812 08:00:55.382483  8347 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: No bootstrap required, opened a new log
I20260812 08:00:55.382691  8347 ts_tablet_manager.cc:1403] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Time spent bootstrapping tablet: real 0.012s	user 0.005s	sys 0.000s
I20260812 08:00:55.382604  8344 consensus_queue.cc:260] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:00:55.383502  8344 ts_tablet_manager.cc:1434] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:00:55.383734  8352 tablet_bootstrap.cc:654] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:55.383844  8301 heartbeater.cc:499] Master 127.5.142.190:44379 was elected leader, sending a full tablet report...
I20260812 08:00:55.384523  8352 log.cc:826] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:55.385756  8347 raft_consensus.cc:359] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:00:55.385982  8347 raft_consensus.cc:385] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:55.386037  8347 raft_consensus.cc:740] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faafbc72cbd444ff90977a2cdb091b72, State: Initialized, Role: FOLLOWER
I20260812 08:00:55.386186  8358 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df: Bootstrap starting.
I20260812 08:00:55.386528  8342 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: No bootstrap required, opened a new log
I20260812 08:00:55.386639  8347 consensus_queue.cc:260] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:00:55.386786  8342 ts_tablet_manager.cc:1403] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Time spent bootstrapping tablet: real 0.015s	user 0.005s	sys 0.000s
I20260812 08:00:55.387495  8347 ts_tablet_manager.cc:1434] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:00:55.387820  7684 heartbeater.cc:499] Master 127.5.142.190:44379 was elected leader, sending a full tablet report...
W20260812 08:00:55.389518  7256 tablet.cc:2406] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:00:55.390074  8352 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c: No bootstrap required, opened a new log
I20260812 08:00:55.391368  8341 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: No bootstrap required, opened a new log
I20260812 08:00:55.391398  8352 ts_tablet_manager.cc:1403] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c: Time spent bootstrapping tablet: real 0.010s	user 0.005s	sys 0.000s
I20260812 08:00:55.391569  8341 ts_tablet_manager.cc:1403] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Time spent bootstrapping tablet: real 0.018s	user 0.002s	sys 0.002s
I20260812 08:00:55.391388  8342 raft_consensus.cc:359] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:00:55.391645  8342 raft_consensus.cc:385] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:55.391716  8342 raft_consensus.cc:740] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1664e32d3cd0473592085aa522379f22, State: Initialized, Role: FOLLOWER
I20260812 08:00:55.392244  8342 consensus_queue.cc:260] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:00:55.393268  8342 ts_tablet_manager.cc:1434] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260812 08:00:55.393627  7874 heartbeater.cc:499] Master 127.5.142.190:44379 was elected leader, sending a full tablet report...
I20260812 08:00:55.394732  8341 raft_consensus.cc:359] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:00:55.394996  8341 raft_consensus.cc:385] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:55.395063  8341 raft_consensus.cc:740] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae3f7a6e9f943e996a6dcea504878d0, State: Initialized, Role: FOLLOWER
I20260812 08:00:55.395414  8352 raft_consensus.cc:359] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:00:55.395529  8341 consensus_queue.cc:260] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:00:55.395762  8352 raft_consensus.cc:385] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:55.395880  8352 raft_consensus.cc:740] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16901c5559a748a9aa770d73e254099c, State: Initialized, Role: FOLLOWER
I20260812 08:00:55.396405  8341 ts_tablet_manager.cc:1434] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.003s
I20260812 08:00:55.396358  8352 consensus_queue.cc:260] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:00:55.397511  8352 ts_tablet_manager.cc:1434] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260812 08:00:55.397939  7461 heartbeater.cc:499] Master 127.5.142.190:44379 was elected leader, sending a full tablet report...
I20260812 08:00:55.400558  8358 tablet_bootstrap.cc:654] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df: Neither blocks nor log segments found. Creating new log.
I20260812 08:00:55.401557  8358 log.cc:826] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df: Log is configured to *not* fsync() on all Append() calls
I20260812 08:00:55.403389  8358 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df: No bootstrap required, opened a new log
I20260812 08:00:55.403577  8358 ts_tablet_manager.cc:1403] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df: Time spent bootstrapping tablet: real 0.018s	user 0.002s	sys 0.002s
I20260812 08:00:55.406068  8358 raft_consensus.cc:359] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:00:55.406248  8358 raft_consensus.cc:385] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:00:55.406287  8358 raft_consensus.cc:740] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 780d7e64a8214418a371f35351b113df, State: Initialized, Role: FOLLOWER
I20260812 08:00:55.406623  8358 consensus_queue.cc:260] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:00:55.407232  8086 heartbeater.cc:499] Master 127.5.142.190:44379 was elected leader, sending a full tablet report...
I20260812 08:00:55.407363  8358 ts_tablet_manager.cc:1434] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df: Time spent starting tablet: real 0.004s	user 0.002s	sys 0.003s
W20260812 08:00:55.427178  7875 tablet.cc:2406] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:00:55.460817  7040 tablet.cc:2406] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:00:55.502341  7685 tablet.cc:2406] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:00:55.517318  8372 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:00:55.517539  8372 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:00:55.519091  8372 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:00:55.524952  7603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:00:55.525292  7603 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16901c5559a748a9aa770d73e254099c in term 0.
I20260812 08:00:55.525755  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:00:55.526019  8008 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16901c5559a748a9aa770d73e254099c in term 0.
I20260812 08:00:55.526585  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:00:55.526836  7804 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16901c5559a748a9aa770d73e254099c in term 0.
I20260812 08:00:55.526962  6968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:00:55.527206  6968 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16901c5559a748a9aa770d73e254099c in term 0.
I20260812 08:00:55.527233  7286 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 780d7e64a8214418a371f35351b113df, faafbc72cbd444ff90977a2cdb091b72; no voters: 
I20260812 08:00:55.527534  8372 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:00:55.527603  8372 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:00:55.527639  8372 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:55.528589  7188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:00:55.528781  8372 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:00:55.528858  7188 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16901c5559a748a9aa770d73e254099c in term 0.
I20260812 08:00:55.529841  8226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:00:55.529917  6968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0"
I20260812 08:00:55.530036  6968 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:55.530092  8226 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16901c5559a748a9aa770d73e254099c in term 0.
I20260812 08:00:55.530246  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22"
I20260812 08:00:55.530368  7804 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:55.530354  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:00:55.530495  8008 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:55.530615  7188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a"
I20260812 08:00:55.530726  7188 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:55.531195  6968 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16901c5559a748a9aa770d73e254099c in term 1.
I20260812 08:00:55.529639  8372 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 1 election: Requested vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:00:55.531605  8008 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16901c5559a748a9aa770d73e254099c in term 1.
I20260812 08:00:55.531795  7188 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16901c5559a748a9aa770d73e254099c in term 1.
I20260812 08:00:55.531901  7804 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16901c5559a748a9aa770d73e254099c in term 1.
I20260812 08:00:55.532063  7603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72"
I20260812 08:00:55.532191  7286 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, 780d7e64a8214418a371f35351b113df; no voters: 
I20260812 08:00:55.532193  7603 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:55.532502  8372 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:00:55.532655  8226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
I20260812 08:00:55.532723  8372 raft_consensus.cc:697] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 1 LEADER]: Becoming Leader. State: Replica: 16901c5559a748a9aa770d73e254099c, State: Running, Role: LEADER
I20260812 08:00:55.532776  8226 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:00:55.533116  8372 consensus_queue.cc:237] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:00:55.533344  7603 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16901c5559a748a9aa770d73e254099c in term 1.
I20260812 08:00:55.533843  8226 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16901c5559a748a9aa770d73e254099c in term 1.
I20260812 08:00:55.536842  6753 catalog_manager.cc:5719] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c reported cstate change: term changed from 0 to 1, leader changed from <none> to 16901c5559a748a9aa770d73e254099c (127.5.142.131). New cstate: current_term: 1 leader_uuid: "16901c5559a748a9aa770d73e254099c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } health_report { overall_health: UNKNOWN } } }
W20260812 08:00:55.550789  8302 tablet.cc:2406] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:00:55.568701  7463 tablet.cc:2406] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:00:55.581369  5690 external_mini_cluster.cc:955] 7 TS(s) registered with all masters
I20260812 08:00:55.582818  5690 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 3ae3f7a6e9f943e996a6dcea504878d0 to finish bootstrapping
I20260812 08:00:55.588086  5690 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 56b60424ed9f4a079689ad82a318d29a to finish bootstrapping
I20260812 08:00:55.592434  5690 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 16901c5559a748a9aa770d73e254099c to finish bootstrapping
I20260812 08:00:55.596993  5690 ts_itest-base.cc:246] Waiting for 1 tablets on tserver faafbc72cbd444ff90977a2cdb091b72 to finish bootstrapping
I20260812 08:00:55.602205  5690 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 1664e32d3cd0473592085aa522379f22 to finish bootstrapping
W20260812 08:00:55.603811  8087 tablet.cc:2406] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:00:55.607364  5690 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 780d7e64a8214418a371f35351b113df to finish bootstrapping
I20260812 08:00:55.611922  5690 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 6801d45ca6614156906e5d571ebcb8a6 to finish bootstrapping
I20260812 08:00:55.635660  7804 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 1 FOLLOWER]: Refusing update from remote peer 16901c5559a748a9aa770d73e254099c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:00:55.635653  8008 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 1 FOLLOWER]: Refusing update from remote peer 16901c5559a748a9aa770d73e254099c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:00:55.635730  6968 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 1 FOLLOWER]: Refusing update from remote peer 16901c5559a748a9aa770d73e254099c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:00:55.635731  8226 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 1 FOLLOWER]: Refusing update from remote peer 16901c5559a748a9aa770d73e254099c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:00:55.636956  8372 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:00:55.637135  8386 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:00:55.637280  8405 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:00:55.637738  8372 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:00:55.638049  7188 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 1 FOLLOWER]: Refusing update from remote peer 16901c5559a748a9aa770d73e254099c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:00:55.642261  8386 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:00:55.647785  7603 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 1 FOLLOWER]: Refusing update from remote peer 16901c5559a748a9aa770d73e254099c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:00:55.653812  8405 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [LEADER]: Connected to new peer: Peer: permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:00:55.661923  8420 mvcc.cc:204] Tried to move back new op lower bound from 7317592701476061184 to 7317592701068644352. Current Snapshot: MvccSnapshot[applied={T|T < 7317592701476061184}]
I20260812 08:00:55.663857  8410 mvcc.cc:204] Tried to move back new op lower bound from 7317592701476061184 to 7317592701068644352. Current Snapshot: MvccSnapshot[applied={T|T < 7317592701476061184}]
I20260812 08:00:55.667994  8423 mvcc.cc:204] Tried to move back new op lower bound from 7317592701506760704 to 7317592701068644352. Current Snapshot: MvccSnapshot[applied={T|T < 7317592701476061184}]
I20260812 08:00:55.671084  8425 mvcc.cc:204] Tried to move back new op lower bound from 7317592701506760704 to 7317592701068644352. Current Snapshot: MvccSnapshot[applied={T|T < 7317592701476061184}]
I20260812 08:00:55.679423  8411 mvcc.cc:204] Tried to move back new op lower bound from 7317592701506760704 to 7317592701068644352. Current Snapshot: MvccSnapshot[applied={T|T < 7317592701476061184}]
I20260812 08:00:55.682694  8430 mvcc.cc:204] Tried to move back new op lower bound from 7317592701506760704 to 7317592701068644352. Current Snapshot: MvccSnapshot[applied={T|T < 7317592701476061184}]
I20260812 08:00:58.008381  8502 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 16901c5559a748a9aa770d73e254099c)
I20260812 08:00:58.008530  8502 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:00:58.009997  8502 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183)
I20260812 08:00:58.014734  7603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 2 candidate_status { last_received { term: 1 index: 219 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:00:58.014957  6966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 2 candidate_status { last_received { term: 1 index: 219 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:00:58.015969  7188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 2 candidate_status { last_received { term: 1 index: 219 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:00:58.017099  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 2 candidate_status { last_received { term: 1 index: 219 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:00:58.017352  8226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 2 candidate_status { last_received { term: 1 index: 219 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:00:58.017527  7902 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:00:58.017798  8502 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 08:00:58.105469  8519 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 16901c5559a748a9aa770d73e254099c)
I20260812 08:00:58.105643  8519 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:00:58.107148  8519 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:00:58.112592  6966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 252 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:00:58.112592  7603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 252 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:00:58.114212  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 252 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:00:58.114400  8008 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6801d45ca6614156906e5d571ebcb8a6 in term 1.
I20260812 08:00:58.114570  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 252 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:00:58.114735  7188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 252 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:00:58.115116  8116 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:00:58.115398  8519 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 08:00:58.356009  8529 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 16901c5559a748a9aa770d73e254099c)
I20260812 08:00:58.356042  8532 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 16901c5559a748a9aa770d73e254099c)
I20260812 08:00:58.356172  8529 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:00:58.356184  8532 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:00:58.357705  8529 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:00:58.360327  8532 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:00:58.363168  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 345 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:00:58.363343  8008 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 1.
I20260812 08:00:58.363765  8226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 345 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:00:58.363924  8226 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 1.
I20260812 08:00:58.366811  7603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 345 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:00:58.366904  6966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 2 candidate_status { last_received { term: 1 index: 345 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:00:58.366956  7603 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 1.
I20260812 08:00:58.367049  6966 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 1.
I20260812 08:00:58.367349  6867 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 3ae3f7a6e9f943e996a6dcea504878d0, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df, faafbc72cbd444ff90977a2cdb091b72; no voters: 
I20260812 08:00:58.367625  8529 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 08:00:58.367681  8529 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 16901c5559a748a9aa770d73e254099c)
I20260812 08:00:58.367666  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 2 candidate_status { last_received { term: 1 index: 345 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:00:58.367719  8529 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:00:58.367782  8008 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 1.
I20260812 08:00:58.368870  8529 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:00:58.369582  8529 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 2 election: Requested vote from peers faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:00:58.369733  8226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 2 candidate_status { last_received { term: 1 index: 345 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:00:58.369866  8226 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 1.
I20260812 08:00:58.370254  7495 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 3ae3f7a6e9f943e996a6dcea504878d0, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df, faafbc72cbd444ff90977a2cdb091b72; no voters: 
I20260812 08:00:58.370476  7603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 345 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72"
I20260812 08:00:58.370530  8532 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 08:00:58.370617  7603 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:00:58.370684  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 345 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:00:58.370776  8008 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:00:58.371613  7603 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 2.
I20260812 08:00:58.371694  8008 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 2.
I20260812 08:00:58.371711  8532 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:00:58.371790  8532 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 2 FOLLOWER]: Advancing to term 3
I20260812 08:00:58.372078  8226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 345 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
I20260812 08:00:58.372193  8226 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:00:58.372835  8532 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:00:58.373056  8226 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 2.
I20260812 08:00:58.373375  6869 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 3ae3f7a6e9f943e996a6dcea504878d0, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df, faafbc72cbd444ff90977a2cdb091b72; no voters: 
I20260812 08:00:58.373358  7188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 345 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:00:58.373468  7187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 345 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a"
I20260812 08:00:58.373567  8529 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 08:00:58.373782  8529 raft_consensus.cc:697] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 2 LEADER]: Becoming Leader. State: Replica: 3ae3f7a6e9f943e996a6dcea504878d0, State: Running, Role: LEADER
I20260812 08:00:58.374130  8529 consensus_queue.cc:237] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 337, Committed index: 337, Last appended: 1.345, Last appended by leader: 345, Current term: 2, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:00:58.374382  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 2 candidate_status { last_received { term: 1 index: 345 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:00:58.374607  6966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 3 candidate_status { last_received { term: 1 index: 345 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0"
I20260812 08:00:58.375495  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 3 candidate_status { last_received { term: 1 index: 345 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22"
I20260812 08:00:58.376446  7187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 2 candidate_status { last_received { term: 1 index: 345 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:00:58.376634  7188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 3 candidate_status { last_received { term: 1 index: 345 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a"
I20260812 08:00:58.377424  8226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 3 candidate_status { last_received { term: 1 index: 345 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
I20260812 08:00:58.377552  8226 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 2 FOLLOWER]: Advancing to term 3
I20260812 08:00:58.377436  6753 catalog_manager.cc:5719] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 reported cstate change: term changed from 1 to 2, leader changed from 16901c5559a748a9aa770d73e254099c (127.5.142.131) to 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129). New cstate: current_term: 2 leader_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } health_report { overall_health: UNKNOWN } } }
I20260812 08:00:58.377665  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 345 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22"
I20260812 08:00:58.377727  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 345 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:00:58.378059  8532 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 3 election: Requested vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:00:58.378511  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 3 candidate_status { last_received { term: 1 index: 345 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:00:58.378580  8226 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 3.
I20260812 08:00:58.378635  8008 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 2 FOLLOWER]: Advancing to term 3
I20260812 08:00:58.379375  8008 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 3.
W20260812 08:00:58.454325  7284 connection.cc:256] RPC call timeout handler was delayed by -1.0471489070914686s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 08:00:58.459682  7603 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 3 FOLLOWER]: Rejecting Update request from peer 16901c5559a748a9aa770d73e254099c for earlier term 1. Current term is 3. Ops: []
I20260812 08:00:58.459874  6966 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 2 LEADER]: Rejecting Update request from peer 16901c5559a748a9aa770d73e254099c for earlier term 1. Current term is 2. Ops: []
I20260812 08:00:58.461160  7375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 2 candidate_status { last_received { term: 1 index: 219 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:00:58.462152  8226 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 3 FOLLOWER]: Rejecting Update request from peer 16901c5559a748a9aa770d73e254099c for earlier term 1. Current term is 3. Ops: [1.253-1.345]
I20260812 08:00:58.465468  8418 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 }, Status: INVALID_TERM, Last received: 1.345, Next index: 346, Last known committed idx: 337, Time since last communication: 0.000s
I20260812 08:00:58.465660  8418 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 }, Status: INVALID_TERM, Last received: 1.252, Next index: 253, Last known committed idx: 244, Time since last communication: 0.000s
I20260812 08:00:58.465859  8418 raft_consensus.cc:3055] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 1 LEADER]: Stepping down as leader of term 1
I20260812 08:00:58.465909  8418 raft_consensus.cc:740] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 16901c5559a748a9aa770d73e254099c, State: Running, Role: LEADER
I20260812 08:00:58.466073  8418 consensus_queue.cc:260] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 341, Committed index: 341, Last appended: 1.345, Last appended by leader: 345, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:00:58.466418  8418 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 1 FOLLOWER]: Advancing to term 3
I20260812 08:00:58.468012  7375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 252 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:00:58.468166  7375 raft_consensus.cc:2368] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6801d45ca6614156906e5d571ebcb8a6 for earlier term 2. Current term is 3.
I20260812 08:00:58.468813  8118 leader_election.cc:400] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183) with higher term. Message: Invalid argument: T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6801d45ca6614156906e5d571ebcb8a6 for earlier term 2. Current term is 3.
I20260812 08:00:58.469233  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 345 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:00:58.469347  7378 raft_consensus.cc:2368] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ae3f7a6e9f943e996a6dcea504878d0 for earlier term 2. Current term is 3.
I20260812 08:00:58.469645  8583 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:40918 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=336, attempt_no=0}) took 1637 ms. Trace:
I20260812 08:00:58.470016  6872 leader_election.cc:400] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 2 election: Vote denied by peer 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183) with higher term. Message: Invalid argument: T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ae3f7a6e9f943e996a6dcea504878d0 for earlier term 2. Current term is 3.
I20260812 08:00:58.470388  8584 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:40918 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=337, attempt_no=0}) took 1634 ms. Trace:
I20260812 08:00:58.470402  7375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 345 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:00:58.470522  7375 raft_consensus.cc:2368] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ae3f7a6e9f943e996a6dcea504878d0 for earlier term 2. Current term is 3.
I20260812 08:00:58.470455  8584 rpcz_store.cc:276] 0812 08:00:56.835943 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:00:56.836111 (+   168us) service_pool.cc:224] Handling call
0812 08:00:58.470360 (+1634249us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:00:56.842736 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:00:56.842803 (+    67us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:00:56.842806 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:00:56.842808 (+     2us) tablet.cc:662] Decoding operations
0812 08:00:56.842885 (+    77us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:00:56.842891 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:00:56.842894 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:00:56.842951 (+    57us) tablet.cc:701] Row locks acquired
0812 08:00:56.842952 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:00:56.842977 (+    25us) write_op.cc:270] Start()
0812 08:00:56.842994 (+    17us) write_op.cc:276] Timestamp: P: 1786521656842944 usec, L: 0
0812 08:00:56.842996 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:00:56.843088 (+    92us) log.cc:844] Serialized 696 byte log entry
0812 08:00:58.458340 (+1615252us) op_driver.cc:464] REPLICATION: finished
0812 08:00:58.469927 (+ 11587us) write_op.cc:301] APPLY: starting
0812 08:00:58.469941 (+    14us) tablet.cc:1367] starting BulkCheckPresence
0812 08:00:58.469971 (+    30us) tablet.cc:1370] finished BulkCheckPresence
0812 08:00:58.469983 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:00:58.470147 (+   164us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:00:58.470175 (+    28us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:00:58.470180 (+     5us) write_op.cc:312] APPLY: finished
0812 08:00:58.470239 (+    59us) log.cc:844] Serialized 77 byte log entry
0812 08:00:58.470280 (+    41us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:00:58.470295 (+    15us) write_op.cc:454] Released schema lock
0812 08:00:58.470300 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":11546,"num_ops":10,"prepare.queue_time_us":6349,"prepare.run_cpu_time_us":382,"prepare.run_wall_time_us":381,"replication_time_us":1615330,"thread_start_us":45,"threads_started":1}]]}
I20260812 08:00:58.470935  6872 leader_election.cc:400] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183) with higher term. Message: Invalid argument: T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ae3f7a6e9f943e996a6dcea504878d0 for earlier term 2. Current term is 3.
I20260812 08:00:58.471390  8584 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:40918 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=338, attempt_no=0}) took 1634 ms. Trace:
I20260812 08:00:58.471450  8584 rpcz_store.cc:276] 0812 08:00:56.836749 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:00:56.836788 (+    39us) service_pool.cc:224] Handling call
0812 08:00:58.471381 (+1634593us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:00:56.843151 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:00:56.843219 (+    68us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:00:56.843222 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:00:56.843223 (+     1us) tablet.cc:662] Decoding operations
0812 08:00:56.843301 (+    78us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:00:56.843306 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:00:56.843309 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:00:56.843368 (+    59us) tablet.cc:701] Row locks acquired
0812 08:00:56.843370 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:00:56.843397 (+    27us) write_op.cc:270] Start()
0812 08:00:56.843417 (+    20us) write_op.cc:276] Timestamp: P: 1786521656843364 usec, L: 0
0812 08:00:56.843419 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:00:56.843557 (+   138us) log.cc:844] Serialized 696 byte log entry
0812 08:00:58.458429 (+1614872us) op_driver.cc:464] REPLICATION: finished
0812 08:00:58.470983 (+ 12554us) write_op.cc:301] APPLY: starting
0812 08:00:58.470992 (+     9us) tablet.cc:1367] starting BulkCheckPresence
0812 08:00:58.471020 (+    28us) tablet.cc:1370] finished BulkCheckPresence
0812 08:00:58.471031 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:00:58.471188 (+   157us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:00:58.471215 (+    27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:00:58.471219 (+     4us) write_op.cc:312] APPLY: finished
0812 08:00:58.471277 (+    58us) log.cc:844] Serialized 77 byte log entry
0812 08:00:58.471315 (+    38us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:00:58.471327 (+    12us) write_op.cc:454] Released schema lock
0812 08:00:58.471331 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":12525,"num_ops":10,"prepare.queue_time_us":6084,"prepare.run_cpu_time_us":471,"prepare.run_wall_time_us":471,"replication_time_us":1614998,"spinlock_wait_cycles":119936,"thread_start_us":45,"threads_started":1}]]}
I20260812 08:00:58.472131  8584 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:40918 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=339, attempt_no=0}) took 1631 ms. Trace:
I20260812 08:00:58.472184  8584 rpcz_store.cc:276] 0812 08:00:56.840711 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:00:56.840754 (+    43us) service_pool.cc:224] Handling call
0812 08:00:58.472110 (+1631356us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:00:56.843654 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:00:56.843724 (+    70us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:00:56.843727 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:00:56.843729 (+     2us) tablet.cc:662] Decoding operations
0812 08:00:56.843823 (+    94us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:00:56.843829 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:00:56.843832 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:00:56.843890 (+    58us) tablet.cc:701] Row locks acquired
0812 08:00:56.843891 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:00:56.843919 (+    28us) write_op.cc:270] Start()
0812 08:00:56.843938 (+    19us) write_op.cc:276] Timestamp: P: 1786521656843885 usec, L: 0
0812 08:00:56.843939 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:00:56.844039 (+   100us) log.cc:844] Serialized 696 byte log entry
0812 08:00:58.458511 (+1614472us) op_driver.cc:464] REPLICATION: finished
0812 08:00:58.471749 (+ 13238us) write_op.cc:301] APPLY: starting
0812 08:00:58.471759 (+    10us) tablet.cc:1367] starting BulkCheckPresence
0812 08:00:58.471784 (+    25us) tablet.cc:1370] finished BulkCheckPresence
0812 08:00:58.471795 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:00:58.471938 (+   143us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:00:58.471963 (+    25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:00:58.471967 (+     4us) write_op.cc:312] APPLY: finished
0812 08:00:58.472017 (+    50us) log.cc:844] Serialized 77 byte log entry
0812 08:00:58.472048 (+    31us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:00:58.472059 (+    11us) write_op.cc:454] Released schema lock
0812 08:00:58.472063 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":13211,"num_ops":10,"prepare.queue_time_us":2590,"prepare.run_cpu_time_us":422,"prepare.run_wall_time_us":421,"replication_time_us":1614561,"thread_start_us":39,"threads_started":1}]]}
I20260812 08:00:58.469794  8583 rpcz_store.cc:276] 0812 08:00:56.832286 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:00:56.832323 (+    37us) service_pool.cc:224] Handling call
0812 08:00:58.469634 (+1637311us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:00:56.842286 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:00:56.842356 (+    70us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:00:56.842359 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:00:56.842360 (+     1us) tablet.cc:662] Decoding operations
0812 08:00:56.842441 (+    81us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:00:56.842446 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:00:56.842449 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:00:56.842504 (+    55us) tablet.cc:701] Row locks acquired
0812 08:00:56.842506 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:00:56.842533 (+    27us) write_op.cc:270] Start()
0812 08:00:56.842557 (+    24us) write_op.cc:276] Timestamp: P: 1786521656842500 usec, L: 0
0812 08:00:56.842559 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:00:56.842677 (+   118us) log.cc:844] Serialized 696 byte log entry
0812 08:00:58.458185 (+1615508us) op_driver.cc:464] REPLICATION: finished
0812 08:00:58.469001 (+ 10816us) write_op.cc:301] APPLY: starting
0812 08:00:58.469017 (+    16us) tablet.cc:1367] starting BulkCheckPresence
0812 08:00:58.469051 (+    34us) tablet.cc:1370] finished BulkCheckPresence
0812 08:00:58.469063 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:00:58.469223 (+   160us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:00:58.469252 (+    29us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:00:58.469256 (+     4us) write_op.cc:312] APPLY: finished
0812 08:00:58.469319 (+    63us) log.cc:844] Serialized 77 byte log entry
0812 08:00:58.469545 (+   226us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:00:58.469560 (+    15us) write_op.cc:454] Released schema lock
0812 08:00:58.469567 (+     7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":10761,"num_ops":10,"prepare.queue_time_us":9713,"prepare.run_cpu_time_us":428,"prepare.run_wall_time_us":427,"replication_time_us":1615596,"spinlock_wait_cycles":2944,"thread_start_us":240,"threads_started":2}]]}
I20260812 08:00:58.472569  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 3 candidate_status { last_received { term: 1 index: 345 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:00:58.473336  7375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 2 candidate_status { last_received { term: 1 index: 345 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:00:58.473779  7378 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 3.
I20260812 08:00:58.464838  8008 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 3 FOLLOWER]: Rejecting Update request from peer 16901c5559a748a9aa770d73e254099c for earlier term 1. Current term is 3. Ops: [1.220-1.345]
W20260812 08:00:58.474882  7317 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:40918: Illegal state: replica 16901c5559a748a9aa770d73e254099c is not leader of this config: current role FOLLOWER
I20260812 08:00:58.475251  7496 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df, faafbc72cbd444ff90977a2cdb091b72; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a
W20260812 08:00:58.475709  7324 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:40918: Illegal state: replica 16901c5559a748a9aa770d73e254099c is not leader of this config: current role FOLLOWER
I20260812 08:00:58.476004  8532 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 3 FOLLOWER]: Leader election won for term 3
W20260812 08:00:58.476047  7321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:40918: Illegal state: replica 16901c5559a748a9aa770d73e254099c is not leader of this config: current role FOLLOWER
I20260812 08:00:58.476238  8532 raft_consensus.cc:697] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 3 LEADER]: Becoming Leader. State: Replica: faafbc72cbd444ff90977a2cdb091b72, State: Running, Role: LEADER
W20260812 08:00:58.476541  7317 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:40918: Illegal state: replica 16901c5559a748a9aa770d73e254099c is not leader of this config: current role FOLLOWER
I20260812 08:00:58.476580  8532 consensus_queue.cc:237] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 337, Committed index: 337, Last appended: 1.345, Last appended by leader: 345, Current term: 3, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:00:58.479714  6753 catalog_manager.cc:5719] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 reported cstate change: term changed from 2 to 3, leader changed from 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129) to faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132). New cstate: current_term: 3 leader_uuid: "faafbc72cbd444ff90977a2cdb091b72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } health_report { overall_health: UNKNOWN } } }
I20260812 08:00:58.488356  8008 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 3 FOLLOWER]: Rejecting Update request from peer 3ae3f7a6e9f943e996a6dcea504878d0 for earlier term 2. Current term is 3. Ops: []
I20260812 08:00:58.489842  7378 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 3 FOLLOWER]: Rejecting Update request from peer 3ae3f7a6e9f943e996a6dcea504878d0 for earlier term 2. Current term is 3. Ops: []
I20260812 08:00:58.490319  8226 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 3 FOLLOWER]: Rejecting Update request from peer 3ae3f7a6e9f943e996a6dcea504878d0 for earlier term 2. Current term is 3. Ops: []
I20260812 08:00:58.491413  8629 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 346, Last known committed idx: 218, Time since last communication: 0.000s
I20260812 08:00:58.491701  7804 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:00:58.491739  8629 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 }, Status: INVALID_TERM, Last received: 0.0, Next index: 346, Last known committed idx: 341, Time since last communication: 0.000s
I20260812 08:00:58.491868  8629 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 }, Status: INVALID_TERM, Last received: 0.0, Next index: 346, Last known committed idx: 244, Time since last communication: 0.000s
I20260812 08:00:58.492034  8629 raft_consensus.cc:3055] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 08:00:58.492085  8629 raft_consensus.cc:740] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ae3f7a6e9f943e996a6dcea504878d0, State: Running, Role: LEADER
I20260812 08:00:58.492249  8629 consensus_queue.cc:260] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 337, Committed index: 337, Last appended: 2.350, Last appended by leader: 350, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:00:58.492571  8629 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 2 FOLLOWER]: Advancing to term 3
I20260812 08:00:58.492961  7804 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 2 FOLLOWER]: Refusing update from remote peer 3ae3f7a6e9f943e996a6dcea504878d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 345. Preceding OpId from leader: term: 2 index: 350. (index mismatch)
I20260812 08:00:58.494019  7603 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 3 LEADER]: Rejecting Update request from peer 3ae3f7a6e9f943e996a6dcea504878d0 for earlier term 2. Current term is 3. Ops: []
I20260812 08:00:58.495831  7187 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:00:58.496965  7187 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 2 FOLLOWER]: Refusing update from remote peer 3ae3f7a6e9f943e996a6dcea504878d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 345. Preceding OpId from leader: term: 2 index: 350. (index mismatch)
I20260812 08:00:58.901196  6966 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 3 FOLLOWER]: Refusing update from remote peer faafbc72cbd444ff90977a2cdb091b72: Log matching property violated. Preceding OpId in replica: term: 2 index: 350. Preceding OpId from leader: term: 3 index: 346. (term mismatch)
I20260812 08:00:58.901292  6966 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting all ops after (but not including) 345
I20260812 08:00:58.901360  6966 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted NO_OP operation due to leader change: 2.346
I20260812 08:00:58.901418  6966 raft_consensus.cc:2889] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260812 08:00:58.901470  6966 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 2.347
W20260812 08:00:58.901672  6966 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38762: Aborted: Op aborted by new leader
I20260812 08:00:58.902184  6966 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 2.348
W20260812 08:00:58.902343  6966 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38762: Aborted: Op aborted by new leader
I20260812 08:00:58.902547  6966 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 2.349
W20260812 08:00:58.902667  6966 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38762: Aborted: Op aborted by new leader
I20260812 08:00:58.902863  6966 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 2.350
W20260812 08:00:58.902978  6966 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38762: Aborted: Op aborted by new leader
I20260812 08:00:58.903635  8596 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 346, Last known committed idx: 337, Time since last communication: 0.000s
I20260812 08:00:58.912348  7187 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 2 FOLLOWER]: Advancing to term 3
I20260812 08:00:58.912441  7378 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 3 FOLLOWER]: Refusing update from remote peer faafbc72cbd444ff90977a2cdb091b72: Log matching property violated. Preceding OpId in replica: term: 1 index: 345. Preceding OpId from leader: term: 3 index: 347. (index mismatch)
I20260812 08:00:58.912820  7804 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 2 FOLLOWER]: Advancing to term 3
I20260812 08:00:58.913177  8226 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 3 FOLLOWER]: Refusing update from remote peer faafbc72cbd444ff90977a2cdb091b72: Log matching property violated. Preceding OpId in replica: term: 1 index: 252. Preceding OpId from leader: term: 3 index: 347. (index mismatch)
I20260812 08:00:58.913600  7187 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 3 FOLLOWER]: Refusing update from remote peer faafbc72cbd444ff90977a2cdb091b72: Log matching property violated. Preceding OpId in replica: term: 1 index: 345. Preceding OpId from leader: term: 3 index: 347. (index mismatch)
I20260812 08:00:58.913609  8532 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 346, Last known committed idx: 341, Time since last communication: 0.000s
I20260812 08:00:58.914043  7804 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 3 FOLLOWER]: Refusing update from remote peer faafbc72cbd444ff90977a2cdb091b72: Log matching property violated. Preceding OpId in replica: term: 1 index: 345. Preceding OpId from leader: term: 3 index: 347. (index mismatch)
I20260812 08:00:58.914068  8639 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 346, Last known committed idx: 244, Time since last communication: 0.000s
I20260812 08:00:58.914461  8008 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 3 FOLLOWER]: Refusing update from remote peer faafbc72cbd444ff90977a2cdb091b72: Log matching property violated. Preceding OpId in replica: term: 1 index: 219. Preceding OpId from leader: term: 3 index: 347. (index mismatch)
I20260812 08:00:58.915328  8596 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 346, Last known committed idx: 341, Time since last communication: 0.000s
I20260812 08:00:58.915598  8596 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 346, Last known committed idx: 218, Time since last communication: 0.000s
I20260812 08:00:58.915958  8635 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 346, Last known committed idx: 341, Time since last communication: 0.000s
I20260812 08:00:58.923584  8581 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:40918 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=340, attempt_no=0}) took 2076 ms. Trace:
I20260812 08:00:58.923669  8581 rpcz_store.cc:276] 0812 08:00:56.847295 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:00:56.847334 (+    39us) service_pool.cc:224] Handling call
0812 08:00:58.923565 (+2076231us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:00:56.849390 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:00:56.849466 (+    76us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:00:56.849469 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:00:56.849471 (+     2us) tablet.cc:662] Decoding operations
0812 08:00:56.849549 (+    78us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:00:56.849553 (+     4us) write_op.cc:641] Partition lock acquired for write op
0812 08:00:56.849556 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:00:56.849598 (+    42us) tablet.cc:701] Row locks acquired
0812 08:00:56.849599 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:00:56.849627 (+    28us) write_op.cc:270] Start()
0812 08:00:56.849657 (+    30us) write_op.cc:276] Timestamp: P: 1786521656849592 usec, L: 0
0812 08:00:56.849658 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:00:56.849876 (+   218us) log.cc:844] Serialized 676 byte log entry
0812 08:00:58.922510 (+2072634us) op_driver.cc:464] REPLICATION: finished
0812 08:00:58.923142 (+   632us) write_op.cc:301] APPLY: starting
0812 08:00:58.923154 (+    12us) tablet.cc:1367] starting BulkCheckPresence
0812 08:00:58.923186 (+    32us) tablet.cc:1370] finished BulkCheckPresence
0812 08:00:58.923200 (+    14us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:00:58.923352 (+   152us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:00:58.923382 (+    30us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:00:58.923386 (+     4us) write_op.cc:312] APPLY: finished
0812 08:00:58.923443 (+    57us) log.cc:844] Serialized 77 byte log entry
0812 08:00:58.923483 (+    40us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:00:58.923498 (+    15us) write_op.cc:454] Released schema lock
0812 08:00:58.923503 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":586,"num_ops":10,"prepare.queue_time_us":1763,"prepare.run_cpu_time_us":522,"prepare.run_wall_time_us":1668,"replication_time_us":2072829,"spinlock_wait_cycles":243456}]]}
I20260812 08:00:58.924466  8581 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:40918 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=341, attempt_no=0}) took 2076 ms. Trace:
I20260812 08:00:58.924526  8581 rpcz_store.cc:276] 0812 08:00:56.847709 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:00:56.847771 (+    62us) service_pool.cc:224] Handling call
0812 08:00:58.924454 (+2076683us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:00:56.851088 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:00:56.851169 (+    81us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:00:56.851172 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:00:56.851174 (+     2us) tablet.cc:662] Decoding operations
0812 08:00:56.851265 (+    91us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:00:56.851271 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:00:56.851274 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:00:56.851339 (+    65us) tablet.cc:701] Row locks acquired
0812 08:00:56.851340 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:00:56.851370 (+    30us) write_op.cc:270] Start()
0812 08:00:56.851390 (+    20us) write_op.cc:276] Timestamp: P: 1786521656851335 usec, L: 0
0812 08:00:56.851392 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:00:56.851523 (+   131us) log.cc:844] Serialized 686 byte log entry
0812 08:00:58.922569 (+2071046us) op_driver.cc:464] REPLICATION: finished
0812 08:00:58.924026 (+  1457us) write_op.cc:301] APPLY: starting
0812 08:00:58.924036 (+    10us) tablet.cc:1367] starting BulkCheckPresence
0812 08:00:58.924077 (+    41us) tablet.cc:1370] finished BulkCheckPresence
0812 08:00:58.924089 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:00:58.924267 (+   178us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:00:58.924294 (+    27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:00:58.924298 (+     4us) write_op.cc:312] APPLY: finished
0812 08:00:58.924348 (+    50us) log.cc:844] Serialized 77 byte log entry
0812 08:00:58.924384 (+    36us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:00:58.924397 (+    13us) write_op.cc:454] Released schema lock
0812 08:00:58.924401 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1431,"num_ops":10,"prepare.queue_time_us":3050,"prepare.run_cpu_time_us":490,"prepare.run_wall_time_us":690,"replication_time_us":2071165,"spinlock_wait_cycles":37760}]]}
I20260812 08:00:58.925343  8582 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:40918 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=343, attempt_no=0}) took 2072 ms. Trace:
I20260812 08:00:58.925396  8582 rpcz_store.cc:276] 0812 08:00:56.853013 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:00:56.853073 (+    60us) service_pool.cc:224] Handling call
0812 08:00:58.925330 (+2072257us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:00:56.853378 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:00:56.853442 (+    64us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:00:56.853444 (+     2us) write_op.cc:435] Acquired schema lock
0812 08:00:56.853445 (+     1us) tablet.cc:662] Decoding operations
0812 08:00:56.853507 (+    62us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:00:56.853511 (+     4us) write_op.cc:641] Partition lock acquired for write op
0812 08:00:56.853513 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:00:56.853562 (+    49us) tablet.cc:701] Row locks acquired
0812 08:00:56.853563 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:00:56.853585 (+    22us) write_op.cc:270] Start()
0812 08:00:56.853600 (+    15us) write_op.cc:276] Timestamp: P: 1786521656853552 usec, L: 0
0812 08:00:56.853601 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:00:56.853707 (+   106us) log.cc:844] Serialized 696 byte log entry
0812 08:00:58.922637 (+2068930us) op_driver.cc:464] REPLICATION: finished
0812 08:00:58.924956 (+  2319us) write_op.cc:301] APPLY: starting
0812 08:00:58.924963 (+     7us) tablet.cc:1367] starting BulkCheckPresence
0812 08:00:58.924988 (+    25us) tablet.cc:1370] finished BulkCheckPresence
0812 08:00:58.924999 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:00:58.925162 (+   163us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:00:58.925187 (+    25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:00:58.925191 (+     4us) write_op.cc:312] APPLY: finished
0812 08:00:58.925241 (+    50us) log.cc:844] Serialized 77 byte log entry
0812 08:00:58.925268 (+    27us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:00:58.925279 (+    11us) write_op.cc:454] Released schema lock
0812 08:00:58.925284 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2287,"num_ops":10,"prepare.queue_time_us":41,"prepare.run_cpu_time_us":377,"prepare.run_wall_time_us":377,"replication_time_us":2069026}]]}
I20260812 08:00:58.931795  8581 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:40918 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=342, attempt_no=0}) took 2084 ms. Trace:
I20260812 08:00:58.931864  8581 rpcz_store.cc:276] 0812 08:00:56.847788 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:00:56.847831 (+    43us) service_pool.cc:224] Handling call
0812 08:00:58.931784 (+2083953us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:00:56.851806 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:00:56.851877 (+    71us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:00:56.851880 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:00:56.851881 (+     1us) tablet.cc:662] Decoding operations
0812 08:00:56.851963 (+    82us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:00:56.851969 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:00:56.851971 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:00:56.852033 (+    62us) tablet.cc:701] Row locks acquired
0812 08:00:56.852035 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:00:56.852062 (+    27us) write_op.cc:270] Start()
0812 08:00:56.852081 (+    19us) write_op.cc:276] Timestamp: P: 1786521656852028 usec, L: 0
0812 08:00:56.852083 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:00:56.852187 (+   104us) log.cc:844] Serialized 696 byte log entry
0812 08:00:58.922605 (+2070418us) op_driver.cc:464] REPLICATION: finished
0812 08:00:58.924826 (+  2221us) write_op.cc:301] APPLY: starting
0812 08:00:58.924836 (+    10us) tablet.cc:1367] starting BulkCheckPresence
0812 08:00:58.924861 (+    25us) tablet.cc:1370] finished BulkCheckPresence
0812 08:00:58.924871 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:00:58.931596 (+  6725us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:00:58.931623 (+    27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:00:58.931627 (+     4us) write_op.cc:312] APPLY: finished
0812 08:00:58.931676 (+    49us) log.cc:844] Serialized 77 byte log entry
0812 08:00:58.931719 (+    43us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:00:58.931732 (+    13us) write_op.cc:454] Released schema lock
0812 08:00:58.931736 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2197,"num_ops":10,"prepare.queue_time_us":3717,"prepare.run_cpu_time_us":433,"prepare.run_wall_time_us":619,"replication_time_us":2070511}]]}
I20260812 08:00:58.945120  8640 mvcc.cc:204] Tried to move back new op lower bound from 7317592714900082688 to 7317592713125191680. Current Snapshot: MvccSnapshot[applied={T|T < 7317592714900082688 or (T in {7317592714900082688})}]
I20260812 08:01:00.060426  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 7466
W20260812 08:01:00.073050  8304 meta_cache.cc:302] tablet f22cf4f6412248aa9e68c6d5d61153e8: replica faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153) has failed: Network error: recv got EOF from 127.5.142.132:43153 (error 108)
W20260812 08:01:00.076576  7284 connection.cc:466] server connection from 127.5.142.132:38567 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.132:38567 (request call id 52) could send its response
W20260812 08:01:00.076805  7905 connection.cc:466] server connection from 127.5.142.132:47893 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.132:47893 (request call id 31) could send its response
W20260812 08:01:00.079424  8117 connection.cc:466] server connection from 127.5.142.132:40517 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.132:40517 (request call id 71) could send its response
W20260812 08:01:00.084225  7076 connection.cc:466] server connection from 127.5.142.132:34117 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.132:34117 (request call id 100) could send its response
I20260812 08:01:01.005635  8691 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:01:01.005808  8691 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:01.006582  8691 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:01.006945  6968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 4 candidate_status { last_received { term: 3 index: 482 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:01.007089  7187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 4 candidate_status { last_received { term: 3 index: 482 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:01.007089  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 4 candidate_status { last_received { term: 3 index: 482 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
W20260812 08:01:01.007441  7284 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.142.132:43153: connect: Connection refused (error 111)
I20260812 08:01:01.007774  8226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 4 candidate_status { last_received { term: 3 index: 482 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:01.007941  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 4 candidate_status { last_received { term: 3 index: 482 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:01.008211  7287 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 16901c5559a748a9aa770d73e254099c; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6
I20260812 08:01:01.008394  8691 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260812 08:01:01.008654  7284 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153): Network error: Client connection negotiation failed: client connection to 127.5.142.132:43153: connect: Connection refused (error 111)
I20260812 08:01:01.115232  8694 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:01:01.115381  8694 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:01.116129  8694 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:01.116601  6968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 521 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:01.116688  7375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 521 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:01.116854  7375 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6801d45ca6614156906e5d571ebcb8a6 in term 3.
I20260812 08:01:01.116837  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 521 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:01.116854  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 521 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:01.117246  7187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 521 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:01.117643  8118 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, 780d7e64a8214418a371f35351b113df
I20260812 08:01:01.117823  8694 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260812 08:01:01.117875  8112 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.142.132:43153: connect: Connection refused (error 111)
W20260812 08:01:01.118963  8112 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153): Network error: Client connection negotiation failed: client connection to 127.5.142.132:43153: connect: Connection refused (error 111)
I20260812 08:01:01.241020  8698 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:01:01.241216  8698 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:01.241273  8700 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:01:01.241415  8700 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:01.242511  8700 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:01.242825  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 4 candidate_status { last_received { term: 3 index: 582 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:01.242810  7187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 4 candidate_status { last_received { term: 3 index: 582 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:01.242959  7804 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 3.
I20260812 08:01:01.243866  7375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 4 candidate_status { last_received { term: 3 index: 582 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:01.244012  7375 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 3.
W20260812 08:01:01.244426  7712 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.142.132:43153: connect: Connection refused (error 111)
I20260812 08:01:01.244442  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 4 candidate_status { last_received { term: 3 index: 582 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
W20260812 08:01:01.244777  6867 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.142.132:43153: connect: Connection refused (error 111)
I20260812 08:01:01.244807  8698 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:01.245396  8226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 4 candidate_status { last_received { term: 3 index: 582 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:01.245554  8226 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 3.
I20260812 08:01:01.246132  6872 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, 6801d45ca6614156906e5d571ebcb8a6; no voters: 56b60424ed9f4a079689ad82a318d29a, 780d7e64a8214418a371f35351b113df
I20260812 08:01:01.246361  8700 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 08:01:01.246454  8700 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 3 FOLLOWER]: Starting leader election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:01:01.247009  8700 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 3 FOLLOWER]: Advancing to term 4
W20260812 08:01:01.247798  6867 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153): Network error: Client connection negotiation failed: client connection to 127.5.142.132:43153: connect: Connection refused (error 111)
I20260812 08:01:01.248276  8700 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:01.248739  7187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 4 candidate_status { last_received { term: 3 index: 582 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
W20260812 08:01:01.249567  7712 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153): Network error: Client connection negotiation failed: client connection to 127.5.142.132:43153: connect: Connection refused (error 111)
I20260812 08:01:01.250587  6968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 4 candidate_status { last_received { term: 3 index: 582 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:01.250928  6968 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1664e32d3cd0473592085aa522379f22 in current term 4: Already voted for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in this term.
I20260812 08:01:01.251861  8226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 4 candidate_status { last_received { term: 3 index: 582 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:01.251995  8224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 4 candidate_status { last_received { term: 3 index: 582 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
I20260812 08:01:01.252104  7375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 4 candidate_status { last_received { term: 3 index: 582 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:01.252236  7375 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 3.
I20260812 08:01:01.252002  8226 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 3.
I20260812 08:01:01.252648  7187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 4 candidate_status { last_received { term: 3 index: 582 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a"
I20260812 08:01:01.252763  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 4 candidate_status { last_received { term: 3 index: 582 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22"
I20260812 08:01:01.252866  7804 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 3 FOLLOWER]: Advancing to term 4
W20260812 08:01:01.252854  6867 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153): Network error: Client connection negotiation failed: client connection to 127.5.142.132:43153: connect: Connection refused (error 111)
I20260812 08:01:01.253093  7375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 4 candidate_status { last_received { term: 3 index: 582 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:01:01.253196  7375 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 3 FOLLOWER]: Advancing to term 4
I20260812 08:01:01.253362  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 4 candidate_status { last_received { term: 3 index: 582 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:01.253723  8700 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 4 election: Requested vote from peers faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:01.253854  7712 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6; no voters: 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, 780d7e64a8214418a371f35351b113df, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:01.254047  7804 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 4.
I20260812 08:01:01.254127  8698 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 08:01:01.254295  7375 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 4.
I20260812 08:01:01.254460  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 4 candidate_status { last_received { term: 3 index: 582 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:01:01.254788  6867 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0; no voters: 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:01.254961  8700 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 08:01:01.574928  8725 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:01:01.575083  8725 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:01.575879  8725 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183)
I20260812 08:01:01.576253  6968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 4 candidate_status { last_received { term: 3 index: 453 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:01.576267  7187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 4 candidate_status { last_received { term: 3 index: 453 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:01.576439  6968 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df in current term 4: Already voted for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in this term.
I20260812 08:01:01.576848  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 4 candidate_status { last_received { term: 3 index: 453 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:01.576967  7375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 4 candidate_status { last_received { term: 3 index: 453 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:01.576973  8226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 4 candidate_status { last_received { term: 3 index: 453 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:01.577080  7804 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df in current term 4: Already voted for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in this term.
I20260812 08:01:01.577129  8226 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df for term 4 because replica has last-logged OpId of term: 3 index: 521, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 453.
I20260812 08:01:01.577137  7375 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df in current term 4: Already voted for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in this term.
W20260812 08:01:01.577298  7898 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.142.132:43153: connect: Connection refused (error 111)
I20260812 08:01:01.577499  7907 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 780d7e64a8214418a371f35351b113df; no voters: 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6
I20260812 08:01:01.577643  8725 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 3 FOLLOWER]: Advancing to term 4
W20260812 08:01:01.578205  7898 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153): Network error: Client connection negotiation failed: client connection to 127.5.142.132:43153: connect: Connection refused (error 111)
I20260812 08:01:01.578611  8725 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 08:01:01.581035  8730 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:01:01.581154  8730 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
W20260812 08:01:01.584050  7074 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.142.132:43153: connect: Connection refused (error 111)
I20260812 08:01:01.584064  8730 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
W20260812 08:01:01.586624  7074 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153): Network error: Client connection negotiation failed: client connection to 127.5.142.132:43153: connect: Connection refused (error 111)
I20260812 08:01:01.587508  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 4 candidate_status { last_received { term: 3 index: 579 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:01.587664  7804 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56b60424ed9f4a079689ad82a318d29a in current term 4: Already voted for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in this term.
I20260812 08:01:01.588126  6968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 4 candidate_status { last_received { term: 3 index: 579 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:01.588320  6968 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56b60424ed9f4a079689ad82a318d29a in current term 4: Already voted for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in this term.
I20260812 08:01:01.588806  7375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 4 candidate_status { last_received { term: 3 index: 579 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:01.588966  7375 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56b60424ed9f4a079689ad82a318d29a in current term 4: Already voted for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in this term.
I20260812 08:01:01.589310  7078 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 56b60424ed9f4a079689ad82a318d29a; no voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:01.589548  8730 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 3 FOLLOWER]: Advancing to term 4
I20260812 08:01:01.589743  8226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 4 candidate_status { last_received { term: 3 index: 579 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:01.589871  8226 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 3.
I20260812 08:01:01.590461  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 4 candidate_status { last_received { term: 3 index: 579 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:01.590592  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 4.
I20260812 08:01:01.590606  8730 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 08:01:03.123963  8748 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:01:03.124126  8748 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:03.124917  8748 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:03.125321  7187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 5 candidate_status { last_received { term: 3 index: 582 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:03.125487  7187 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 4.
I20260812 08:01:03.125427  6968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 5 candidate_status { last_received { term: 3 index: 582 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:03.125618  6968 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 4.
I20260812 08:01:03.125622  7375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 5 candidate_status { last_received { term: 3 index: 582 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:03.125754  7375 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 4.
I20260812 08:01:03.125739  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 5 candidate_status { last_received { term: 3 index: 582 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:03.125984  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 4.
I20260812 08:01:03.126082  8226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 5 candidate_status { last_received { term: 3 index: 582 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:03.126227  8226 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 3.
I20260812 08:01:03.126397  7715 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a; no voters: 
I20260812 08:01:03.126864  8748 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 08:01:03.126919  8748 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:01:03.126947  8748 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 4 FOLLOWER]: Advancing to term 5
W20260812 08:01:03.126996  7712 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153): Network error: Client connection negotiation failed: client connection to 127.5.142.132:43153: connect: Connection refused (error 111)
I20260812 08:01:03.127862  8748 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:03.128446  8748 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 5 election: Requested vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:03.128645  6968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 5 candidate_status { last_received { term: 3 index: 582 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0"
I20260812 08:01:03.128697  7187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 5 candidate_status { last_received { term: 3 index: 582 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a"
I20260812 08:01:03.128775  6968 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 4 FOLLOWER]: Advancing to term 5
I20260812 08:01:03.128810  7187 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 4 FOLLOWER]: Advancing to term 5
I20260812 08:01:03.128800  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 5 candidate_status { last_received { term: 3 index: 582 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:01:03.128923  8004 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 4 FOLLOWER]: Advancing to term 5
I20260812 08:01:03.129386  7375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 5 candidate_status { last_received { term: 3 index: 582 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:01:03.129505  7375 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 4 FOLLOWER]: Advancing to term 5
I20260812 08:01:03.129860  7187 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 5.
I20260812 08:01:03.129860  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 5.
I20260812 08:01:03.129992  6968 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 5.
I20260812 08:01:03.130272  8226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 5 candidate_status { last_received { term: 3 index: 582 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
I20260812 08:01:03.130399  8226 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 3 FOLLOWER]: Advancing to term 5
I20260812 08:01:03.130676  7375 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 5.
I20260812 08:01:03.130925  7715 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, 780d7e64a8214418a371f35351b113df; no voters: 
I20260812 08:01:03.131189  8748 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 5 FOLLOWER]: Leader election won for term 5
W20260812 08:01:03.131259  7712 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153): Network error: Client connection negotiation failed: client connection to 127.5.142.132:43153: connect: Connection refused (error 111)
I20260812 08:01:03.131395  8748 raft_consensus.cc:697] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 5 LEADER]: Becoming Leader. State: Replica: 1664e32d3cd0473592085aa522379f22, State: Running, Role: LEADER
I20260812 08:01:03.131464  8226 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 5.
I20260812 08:01:03.131804  8748 consensus_queue.cc:237] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 574, Committed index: 574, Last appended: 3.582, Last appended by leader: 582, Current term: 5, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:03.134529  6753 catalog_manager.cc:5719] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 reported cstate change: term changed from 3 to 5, leader changed from faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132) to 1664e32d3cd0473592085aa522379f22 (127.5.142.133). New cstate: current_term: 5 leader_uuid: "1664e32d3cd0473592085aa522379f22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } health_report { overall_health: UNKNOWN } } }
W20260812 08:01:03.526053  7712 consensus_peers.cc:597] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 -> Peer faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153): Couldn't send request to peer faafbc72cbd444ff90977a2cdb091b72. Status: Network error: Client connection negotiation failed: client connection to 127.5.142.132:43153: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 08:01:03.547492  8004 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 5 FOLLOWER]: Refusing update from remote peer 1664e32d3cd0473592085aa522379f22: Log matching property violated. Preceding OpId in replica: term: 3 index: 453. Preceding OpId from leader: term: 5 index: 583. (index mismatch)
I20260812 08:01:03.548110  8748 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 583, Last known committed idx: 447, Time since last communication: 0.000s
I20260812 08:01:03.587412  7187 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 5 FOLLOWER]: Refusing update from remote peer 1664e32d3cd0473592085aa522379f22: Log matching property violated. Preceding OpId in replica: term: 3 index: 579. Preceding OpId from leader: term: 5 index: 583. (index mismatch)
I20260812 08:01:03.588102  8748 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 583, Last known committed idx: 571, Time since last communication: 0.000s
I20260812 08:01:03.594507  8226 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 5 FOLLOWER]: Refusing update from remote peer 1664e32d3cd0473592085aa522379f22: Log matching property violated. Preceding OpId in replica: term: 3 index: 521. Preceding OpId from leader: term: 5 index: 583. (index mismatch)
I20260812 08:01:03.595116  8748 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 583, Last known committed idx: 520, Time since last communication: 0.000s
I20260812 08:01:03.601944  7375 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 5 FOLLOWER]: Refusing update from remote peer 1664e32d3cd0473592085aa522379f22: Log matching property violated. Preceding OpId in replica: term: 3 index: 482. Preceding OpId from leader: term: 5 index: 583. (index mismatch)
I20260812 08:01:03.602969  6968 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 5 FOLLOWER]: Refusing update from remote peer 1664e32d3cd0473592085aa522379f22: Log matching property violated. Preceding OpId in replica: term: 3 index: 582. Preceding OpId from leader: term: 5 index: 583. (index mismatch)
I20260812 08:01:03.603588  8751 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 583, Last known committed idx: 477, Time since last communication: 0.000s
I20260812 08:01:03.604748  8748 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 583, Last known committed idx: 574, Time since last communication: 0.000s
I20260812 08:01:03.633996  8812 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38762 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=571, attempt_no=1}) took 3550 ms. Trace:
I20260812 08:01:03.634130  8812 rpcz_store.cc:276] 0812 08:01:00.083746 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:00.084332 (+   586us) service_pool.cc:224] Handling call
0812 08:01:03.633970 (+3549638us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:01:03.635295  8812 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38762 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=573, attempt_no=1}) took 3551 ms. Trace:
I20260812 08:01:03.635413  8812 rpcz_store.cc:276] 0812 08:01:00.083901 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:00.084391 (+   490us) service_pool.cc:224] Handling call
0812 08:01:03.635280 (+3550889us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:01:03.636297  8812 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38762 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=574, attempt_no=1}) took 3552 ms. Trace:
I20260812 08:01:03.636390  8812 rpcz_store.cc:276] 0812 08:01:00.083846 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:00.084166 (+   320us) service_pool.cc:224] Handling call
0812 08:01:03.636285 (+3552119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:01:03.637200  8812 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38762 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=576, attempt_no=1}) took 3553 ms. Trace:
W20260812 08:01:03.637477  6893 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38762: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
I20260812 08:01:03.637735  8812 rpcz_store.cc:276] 0812 08:01:00.083796 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:00.084008 (+   212us) service_pool.cc:224] Handling call
0812 08:01:03.637189 (+3553181us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260812 08:01:03.637629  6899 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38762: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:03.638007  6905 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38762: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
I20260812 08:01:03.638362  8812 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38762 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=575, attempt_no=1}) took 3554 ms. Trace:
I20260812 08:01:03.638453  8812 rpcz_store.cc:276] 0812 08:01:00.083673 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:00.084252 (+   579us) service_pool.cc:224] Handling call
0812 08:01:03.638351 (+3554099us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4480}
I20260812 08:01:03.639351  8812 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38762 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=578, attempt_no=1}) took 3556 ms. Trace:
I20260812 08:01:03.639438  8812 rpcz_store.cc:276] 0812 08:01:00.083263 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:00.083943 (+   680us) service_pool.cc:224] Handling call
0812 08:01:03.639340 (+3555397us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:01:03.640066  8812 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38762 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=577, attempt_no=1}) took 3556 ms. Trace:
I20260812 08:01:03.640156  8812 rpcz_store.cc:276] 0812 08:01:00.083339 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:00.084317 (+   978us) service_pool.cc:224] Handling call
0812 08:01:03.640055 (+3555738us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:01:03.640794  8812 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38762 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=579, attempt_no=1}) took 3557 ms. Trace:
I20260812 08:01:03.640879  8812 rpcz_store.cc:276] 0812 08:01:00.083400 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:00.084235 (+   835us) service_pool.cc:224] Handling call
0812 08:01:03.640783 (+3556548us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260812 08:01:03.641776  6905 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38762: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:03.644568  6899 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38762: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:03.645138  6905 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38762: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
I20260812 08:01:03.764227  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.132:43153
--local_ip_for_outbound_sockets=127.5.142.132
--tserver_master_addrs=127.5.142.190:44379
--webserver_port=43191
--webserver_interface=127.5.142.132
--builtin_ntp_servers=127.5.142.148:43055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
W20260812 08:01:04.001267  8841 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:04.001663  8841 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:04.001803  8841 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:04.006994  8841 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:04.007216  8841 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.132
I20260812 08:01:04.012481  8841 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:43055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.132:43153
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.5.142.132
--webserver_port=43191
--tserver_master_addrs=127.5.142.190:44379
--never_fsync=true
--heap_profile_path=/tmp/kudu.8841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.132

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:04.015067  8841 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:04.016764  8841 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:04.027954  8855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:04.028363  8853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:04.051873  8841 server_base.cc:1061] running on GCE node
W20260812 08:01:04.069976  8852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:04.070719  8841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:04.071576  8841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:04.074862  8841 hybrid_clock.cc:648] HybridClock initialized: now 1786521664074833 us; error 37 us; skew 500 ppm
I20260812 08:01:04.077369  8841 webserver.cc:533] Webserver started at http://127.5.142.132:43191/ using document root <none> and password file <none>
I20260812 08:01:04.078186  8841 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:04.078281  8841 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:04.091619  8841 fs_manager.cc:714] Time spent opening directory manager: real 0.010s	user 0.002s	sys 0.001s
I20260812 08:01:04.097612  8870 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:04.099321  8841 fs_manager.cc:730] Time spent opening block manager: real 0.006s	user 0.001s	sys 0.001s
I20260812 08:01:04.099474  8841 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "faafbc72cbd444ff90977a2cdb091b72"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-ncp9"
I20260812 08:01:04.100033  8841 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:04.120448  8841 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:04.122862  8841 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:04.123189  8841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:04.123972  8841 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:04.138432  8880 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:01:04.162303  8841 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:01:04.162400  8841 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.036s	user 0.001s	sys 0.000s
I20260812 08:01:04.162473  8841 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:01:04.167380  8841 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:01:04.167550  8841 ts_tablet_manager.cc:595] Time spent register tablets: real 0.005s	user 0.003s	sys 0.001s
I20260812 08:01:04.195211  8841 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.132:43153
I20260812 08:01:04.197728  8841 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20260812 08:01:04.198766  9039 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.132:43153 every 8 connection(s)
I20260812 08:01:04.199255  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 8841
I20260812 08:01:04.273205  8880 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Bootstrap starting.
I20260812 08:01:04.372139  9040 heartbeater.cc:344] Connected to a master server at 127.5.142.190:44379
I20260812 08:01:04.372471  9040 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:04.373248  9040 heartbeater.cc:507] Master 127.5.142.190:44379 requested a full tablet report, sending...
I20260812 08:01:04.375952  6748 ts_manager.cc:194] Re-registered known tserver with Master: faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153)
I20260812 08:01:04.377013  6748 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.132:36345
I20260812 08:01:04.456503  8880 log.cc:826] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:04.947242  8880 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Bootstrap replayed 1/1 log segments. Stats: ops{read=582 overwritten=0 applied=574 ignored=0} inserts{seen=5720 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 8 replicates
I20260812 08:01:04.947752  8880 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Bootstrap complete.
I20260812 08:01:04.948863  8880 ts_tablet_manager.cc:1403] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Time spent bootstrapping tablet: real 0.764s	user 0.506s	sys 0.032s
I20260812 08:01:04.950968  8880 raft_consensus.cc:359] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 3 FOLLOWER]: Replica starting. Triggering 8 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:04.953339  8880 raft_consensus.cc:740] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: faafbc72cbd444ff90977a2cdb091b72, State: Initialized, Role: FOLLOWER
I20260812 08:01:04.953907  8880 consensus_queue.cc:260] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 574, Last appended: 3.582, Last appended by leader: 582, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:04.954813  9040 heartbeater.cc:499] Master 127.5.142.190:44379 was elected leader, sending a full tablet report...
I20260812 08:01:04.954854  8880 ts_tablet_manager.cc:1434] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260812 08:01:04.957734  8968 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 3 FOLLOWER]: Advancing to term 5
W20260812 08:01:05.261190  8782 log.cc:927] Time spent T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6: Append to log took a long time: real 0.458s	user 0.000s	sys 0.000s
I20260812 08:01:05.283890  9053 mvcc.cc:204] Tried to move back new op lower bound from 7317592739119714304 to 7317592732191764480. Current Snapshot: MvccSnapshot[applied={T|T < 7317592736506114048 or (T in {7317592736509628416,7317592736517443584,7317592736519606272,7317592736576393216,7317592736578678784,7317592736606310400,7317592736618381312,7317592736644689920,7317592736642658304,7317592736652615680,7317592736745160704,7317592736754057216,7317592736775340032})}]
I20260812 08:01:06.303059  9073 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1664e32d3cd0473592085aa522379f22)
I20260812 08:01:06.303212  9073 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:06.303942  9073 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:06.304358  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 858 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:06.304587  6968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 858 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:06.304819  7186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 858 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:06.305470  8006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 858 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:06.305922  8112 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 6801d45ca6614156906e5d571ebcb8a6; no voters: 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, 780d7e64a8214418a371f35351b113df
I20260812 08:01:06.306123  9073 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 08:01:06.308689  8968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 858 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:06.525890  9076 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1664e32d3cd0473592085aa522379f22)
I20260812 08:01:06.526105  9076 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:06.527591  9076 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:06.528774  9089 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1664e32d3cd0473592085aa522379f22)
I20260812 08:01:06.528995  9089 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:06.531239  7186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 6 candidate_status { last_received { term: 5 index: 874 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:06.532128  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 6 candidate_status { last_received { term: 5 index: 874 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:06.532181  8006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 6 candidate_status { last_received { term: 5 index: 874 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:06.532749  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 6 candidate_status { last_received { term: 5 index: 874 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:06.532917  8225 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 5.
I20260812 08:01:06.533260  9089 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:06.533581  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 6 candidate_status { last_received { term: 5 index: 871 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:06.534904  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 6 candidate_status { last_received { term: 5 index: 871 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:06.535036  8225 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 5.
I20260812 08:01:06.536491  7186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 6 candidate_status { last_received { term: 5 index: 871 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:06.537017  8968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 6 candidate_status { last_received { term: 5 index: 874 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:06.537181  8968 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 5.
I20260812 08:01:06.537592  8006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 6 candidate_status { last_received { term: 5 index: 871 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:06.537755  6968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 6 candidate_status { last_received { term: 5 index: 871 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:06.537916  6968 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate faafbc72cbd444ff90977a2cdb091b72 for term 6 because replica has last-logged OpId of term: 5 index: 874, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 871.
I20260812 08:01:06.538229  8875 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72; no voters: 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, 780d7e64a8214418a371f35351b113df
I20260812 08:01:06.538463  9076 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 08:01:06.761001  9094 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1664e32d3cd0473592085aa522379f22)
I20260812 08:01:06.761162  9094 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:06.761869  9094 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183)
I20260812 08:01:06.762343  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 6 candidate_status { last_received { term: 5 index: 788 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:06.762547  7378 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df for term 6 because replica has last-logged OpId of term: 5 index: 819, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 788.
I20260812 08:01:06.762609  7186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 6 candidate_status { last_received { term: 5 index: 788 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:06.762655  6968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 6 candidate_status { last_received { term: 5 index: 788 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:06.762841  6968 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df for term 6 because replica has last-logged OpId of term: 5 index: 874, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 788.
I20260812 08:01:06.763051  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 6 candidate_status { last_received { term: 5 index: 788 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:06.763222  8225 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df for term 6 because replica has last-logged OpId of term: 5 index: 858, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 788.
I20260812 08:01:06.763402  9096 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1664e32d3cd0473592085aa522379f22)
I20260812 08:01:06.763552  7905 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 780d7e64a8214418a371f35351b113df; no voters: 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6
I20260812 08:01:06.763558  9096 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:06.763751  9094 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 08:01:06.764077  9098 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1664e32d3cd0473592085aa522379f22)
I20260812 08:01:06.764241  9098 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:06.764364  6968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 6 candidate_status { last_received { term: 5 index: 819 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:06.764547  6968 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16901c5559a748a9aa770d73e254099c for term 6 because replica has last-logged OpId of term: 5 index: 874, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 819.
I20260812 08:01:06.765377  9096 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:06.765384  6968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 6 candidate_status { last_received { term: 5 index: 866 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:06.765508  6968 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56b60424ed9f4a079689ad82a318d29a for term 6 because replica has last-logged OpId of term: 5 index: 874, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 866.
I20260812 08:01:06.765574  7186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 6 candidate_status { last_received { term: 5 index: 819 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:06.765753  7186 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16901c5559a748a9aa770d73e254099c for term 6 because replica has last-logged OpId of term: 5 index: 866, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 819.
I20260812 08:01:06.765751  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 6 candidate_status { last_received { term: 5 index: 819 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:06.765897  8225 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16901c5559a748a9aa770d73e254099c for term 6 because replica has last-logged OpId of term: 5 index: 858, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 819.
I20260812 08:01:06.766064  8006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 6 candidate_status { last_received { term: 5 index: 819 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:06.766181  8006 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16901c5559a748a9aa770d73e254099c in term 5.
I20260812 08:01:06.767638  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 6 candidate_status { last_received { term: 5 index: 866 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:06.767640  8968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 6 candidate_status { last_received { term: 5 index: 788 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:06.767773  8225 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 5.
I20260812 08:01:06.767818  8968 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df for term 6 because replica has last-logged OpId of term: 5 index: 871, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 788.
I20260812 08:01:06.768076  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 6 candidate_status { last_received { term: 5 index: 866 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:06.768224  7378 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 5.
I20260812 08:01:06.768567  8006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 6 candidate_status { last_received { term: 5 index: 866 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:06.768576  9098 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:06.768694  8006 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 5.
I20260812 08:01:06.768976  7074 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 3ae3f7a6e9f943e996a6dcea504878d0
I20260812 08:01:06.769162  9098 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 08:01:06.769217  9098 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1664e32d3cd0473592085aa522379f22)
I20260812 08:01:06.769294  9098 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 5 FOLLOWER]: Advancing to term 6
I20260812 08:01:06.770427  9098 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:06.771224  8968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 6 candidate_status { last_received { term: 5 index: 819 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:06.771381  8968 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16901c5559a748a9aa770d73e254099c for term 6 because replica has last-logged OpId of term: 5 index: 871, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 819.
I20260812 08:01:06.771687  7284 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 780d7e64a8214418a371f35351b113df; no voters: 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:06.771775  6968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 6 candidate_status { last_received { term: 5 index: 866 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0"
I20260812 08:01:06.771770  8962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 6 candidate_status { last_received { term: 5 index: 866 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72"
I20260812 08:01:06.771876  8962 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 5 FOLLOWER]: Advancing to term 6
I20260812 08:01:06.771876  6968 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 5 FOLLOWER]: Advancing to term 6
I20260812 08:01:06.771936  9096 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 08:01:06.772167  9098 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 6 election: Requested vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:06.772161  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 6 candidate_status { last_received { term: 5 index: 866 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
I20260812 08:01:06.772272  8225 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 5 FOLLOWER]: Advancing to term 6
I20260812 08:01:06.772965  6968 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 56b60424ed9f4a079689ad82a318d29a for term 6 because replica has last-logged OpId of term: 5 index: 874, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 866.
I20260812 08:01:06.772965  8962 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 56b60424ed9f4a079689ad82a318d29a for term 6 because replica has last-logged OpId of term: 5 index: 871, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 866.
I20260812 08:01:06.773077  8225 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 6.
I20260812 08:01:06.773353  8006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 6 candidate_status { last_received { term: 5 index: 866 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:01:06.773478  8006 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 5 FOLLOWER]: Advancing to term 6
I20260812 08:01:06.773500  8968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 6 candidate_status { last_received { term: 5 index: 866 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:06.773559  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 6 candidate_status { last_received { term: 5 index: 866 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:01:06.773624  8968 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56b60424ed9f4a079689ad82a318d29a for term 6 because replica has last-logged OpId of term: 5 index: 871, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 866.
I20260812 08:01:06.773669  7378 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 5 FOLLOWER]: Advancing to term 6
I20260812 08:01:06.774505  8006 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 6.
I20260812 08:01:06.774597  7378 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 6.
I20260812 08:01:06.774887  7078 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 3ae3f7a6e9f943e996a6dcea504878d0, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:06.775085  9098 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 6 FOLLOWER]: Leader election won for term 6
I20260812 08:01:06.775278  9098 raft_consensus.cc:697] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 6 LEADER]: Becoming Leader. State: Replica: 56b60424ed9f4a079689ad82a318d29a, State: Running, Role: LEADER
I20260812 08:01:06.775769  9098 consensus_queue.cc:237] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 866, Committed index: 866, Last appended: 5.866, Last appended by leader: 866, Current term: 6, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:06.778545  6746 catalog_manager.cc:5719] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a reported cstate change: term changed from 5 to 6, leader changed from 1664e32d3cd0473592085aa522379f22 (127.5.142.133) to 56b60424ed9f4a079689ad82a318d29a (127.5.142.130). New cstate: current_term: 6 leader_uuid: "56b60424ed9f4a079689ad82a318d29a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } health_report { overall_health: UNKNOWN } } }
W20260812 08:01:07.112659  7712 connection.cc:256] RPC call timeout handler was delayed by -1.6266947439871728s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 08:01:07.112841  7712 connection.cc:256] RPC call timeout handler was delayed by -1.18839390296489s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 08:01:07.112881  7715 connection.cc:256] RPC call timeout handler was delayed by -1.5221106521785259s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 08:01:07.113333  7804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 6 candidate_status { last_received { term: 5 index: 866 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
W20260812 08:01:07.112826  7714 connection.cc:256] RPC call timeout handler was delayed by -1.4110247241333127s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 08:01:07.112720  7713 connection.cc:256] RPC call timeout handler was delayed by -1.2030858669895679s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 08:01:07.114754  6968 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 6 FOLLOWER]: Rejecting Update request from peer 1664e32d3cd0473592085aa522379f22 for earlier term 5. Current term is 6. Ops: []
I20260812 08:01:07.115041  7787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 6 candidate_status { last_received { term: 5 index: 874 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:07.115393  6868 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 3ae3f7a6e9f943e996a6dcea504878d0, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72; no voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 56b60424ed9f4a079689ad82a318d29a, 780d7e64a8214418a371f35351b113df
I20260812 08:01:07.116784  7790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 858 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:07.117509  7795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 6 candidate_status { last_received { term: 5 index: 788 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:07.117596  9115 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 08:01:07.117944  7797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 6 candidate_status { last_received { term: 5 index: 819 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:07.118371  7801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 6 candidate_status { last_received { term: 5 index: 866 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22"
I20260812 08:01:07.118672  8225 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 6 FOLLOWER]: Rejecting Update request from peer 1664e32d3cd0473592085aa522379f22 for earlier term 5. Current term is 6. Ops: [5.859-5.874]
I20260812 08:01:07.116796  8827 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 }, Status: INVALID_TERM, Last received: 5.874, Next index: 875, Last known committed idx: 866, Time since last communication: 0.003s
I20260812 08:01:07.119778  8827 raft_consensus.cc:3055] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 5 LEADER]: Stepping down as leader of term 5
I20260812 08:01:07.119853  8827 raft_consensus.cc:740] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1664e32d3cd0473592085aa522379f22, State: Running, Role: LEADER
I20260812 08:01:07.120030  8827 consensus_queue.cc:260] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 866, Committed index: 866, Last appended: 5.874, Last appended by leader: 874, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:07.120602  8831 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 }, Status: INVALID_TERM, Last received: 5.858, Next index: 859, Last known committed idx: 851, Time since last communication: 0.000s
I20260812 08:01:07.122920  8827 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 5 FOLLOWER]: Advancing to term 6
I20260812 08:01:07.123654  7378 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 6 FOLLOWER]: Rejecting Update request from peer 1664e32d3cd0473592085aa522379f22 for earlier term 5. Current term is 6. Ops: [5.820-5.874]
I20260812 08:01:07.124560  8004 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 6 FOLLOWER]: Rejecting Update request from peer 1664e32d3cd0473592085aa522379f22 for earlier term 5. Current term is 6. Ops: [5.789-5.874]
I20260812 08:01:07.127081  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 6 candidate_status { last_received { term: 5 index: 871 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:07.127244  7803 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate faafbc72cbd444ff90977a2cdb091b72 for term 6 because replica has last-logged OpId of term: 5 index: 874, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 871.
I20260812 08:01:07.200840  6968 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 6 FOLLOWER]: Refusing update from remote peer 56b60424ed9f4a079689ad82a318d29a: Log matching property violated. Preceding OpId in replica: term: 5 index: 874. Preceding OpId from leader: term: 6 index: 867. (term mismatch)
I20260812 08:01:07.200969  6968 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting all ops after (but not including) 866
I20260812 08:01:07.200994  6968 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 5.867
I20260812 08:01:07.201258  6968 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 5.868
I20260812 08:01:07.201419  6968 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 5.869
I20260812 08:01:07.201562  6968 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 5.870
I20260812 08:01:07.201699  6968 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 5.871
I20260812 08:01:07.201839  6968 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 5.872
I20260812 08:01:07.201980  6968 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 5.873
I20260812 08:01:07.202126  6968 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 5.874
I20260812 08:01:07.202792  9098 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 867, Last known committed idx: 866, Time since last communication: 0.000s
I20260812 08:01:07.205119  7378 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 6 FOLLOWER]: Refusing update from remote peer 56b60424ed9f4a079689ad82a318d29a: Log matching property violated. Preceding OpId in replica: term: 5 index: 819. Preceding OpId from leader: term: 6 index: 867. (index mismatch)
I20260812 08:01:07.205535  9105 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Connected to new peer: Peer: permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 867, Last known committed idx: 818, Time since last communication: 0.000s
I20260812 08:01:07.224359  8225 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 6 FOLLOWER]: Refusing update from remote peer 56b60424ed9f4a079689ad82a318d29a: Log matching property violated. Preceding OpId in replica: term: 5 index: 858. Preceding OpId from leader: term: 6 index: 867. (index mismatch)
I20260812 08:01:07.224807  9105 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 867, Last known committed idx: 851, Time since last communication: 0.000s
I20260812 08:01:07.230271  7803 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 6 FOLLOWER]: Refusing update from remote peer 56b60424ed9f4a079689ad82a318d29a: Log matching property violated. Preceding OpId in replica: term: 5 index: 874. Preceding OpId from leader: term: 6 index: 867. (term mismatch)
I20260812 08:01:07.230372  7803 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting all ops after (but not including) 866
I20260812 08:01:07.230445  7803 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting uncommitted WRITE_OP operation due to leader change: 5.867
W20260812 08:01:07.230656  7803 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45238: Aborted: Op aborted by new leader
I20260812 08:01:07.230656  8968 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 6 FOLLOWER]: Refusing update from remote peer 56b60424ed9f4a079689ad82a318d29a: Log matching property violated. Preceding OpId in replica: term: 5 index: 871. Preceding OpId from leader: term: 6 index: 867. (term mismatch)
I20260812 08:01:07.230759  8968 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting all ops after (but not including) 866
I20260812 08:01:07.230824  8968 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 5.867
I20260812 08:01:07.230918  7803 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:45238 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=863, attempt_no=0}) took 2212 ms. Trace:
I20260812 08:01:07.231103  8968 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 5.868
I20260812 08:01:07.231274  8968 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 5.869
I20260812 08:01:07.231407  8968 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 5.870
I20260812 08:01:07.231544  8968 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 5.871
I20260812 08:01:07.230996  7803 rpcz_store.cc:276] 0812 08:01:05.018258 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:05.018335 (+    77us) service_pool.cc:224] Handling call
0812 08:01:07.230904 (+2212569us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:05.018753 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:05.018843 (+    90us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:05.018846 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:05.018847 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:05.018942 (+    95us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:05.018949 (+     7us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:05.018952 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:05.019018 (+    66us) tablet.cc:701] Row locks acquired
0812 08:01:05.019019 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:05.019049 (+    30us) write_op.cc:270] Start()
0812 08:01:05.019078 (+    29us) write_op.cc:276] Timestamp: P: 1786521665019012 usec, L: 0
0812 08:01:05.019080 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:05.019213 (+   133us) log.cc:844] Serialized 696 byte log entry
0812 08:01:07.230557 (+2211344us) op_driver.cc:464] REPLICATION: finished
0812 08:01:07.230592 (+    35us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 08:01:07.230597 (+     5us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:07.230632 (+    35us) write_op.cc:454] Released schema lock
0812 08:01:07.230639 (+     7us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":55,"prepare.run_cpu_time_us":546,"prepare.run_wall_time_us":546,"raft.queue_time_us":267,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":312,"replication_time_us":2211415,"spinlock_wait_cycles":768}]]}
I20260812 08:01:07.231925  7803 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting uncommitted WRITE_OP operation due to leader change: 5.868
W20260812 08:01:07.232053  7803 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45238: Aborted: Op aborted by new leader
I20260812 08:01:07.232169  7803 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:45238 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=865, attempt_no=0}) took 2213 ms. Trace:
I20260812 08:01:07.232208  7803 rpcz_store.cc:276] 0812 08:01:05.018406 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:05.018709 (+   303us) service_pool.cc:224] Handling call
0812 08:01:07.232160 (+2213451us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:05.019323 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:05.019407 (+    84us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:05.019410 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:05.019411 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:05.019497 (+    86us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:05.019503 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:05.019506 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:05.019567 (+    61us) tablet.cc:701] Row locks acquired
0812 08:01:05.019568 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:05.019596 (+    28us) write_op.cc:270] Start()
0812 08:01:05.019615 (+    19us) write_op.cc:276] Timestamp: P: 1786521665019559 usec, L: 0
0812 08:01:05.019616 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:01:05.019721 (+   105us) log.cc:844] Serialized 696 byte log entry
0812 08:01:07.232012 (+2212291us) op_driver.cc:464] REPLICATION: finished
0812 08:01:07.232018 (+     6us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 08:01:07.232022 (+     4us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:07.232044 (+    22us) write_op.cc:454] Released schema lock
0812 08:01:07.232049 (+     5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":381,"prepare.run_cpu_time_us":434,"prepare.run_wall_time_us":434,"replication_time_us":2212358}]]}
I20260812 08:01:07.232455  7803 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting uncommitted WRITE_OP operation due to leader change: 5.869
W20260812 08:01:07.232578  7803 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45238: Aborted: Op aborted by new leader
I20260812 08:01:07.232678  7803 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:45238 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=866, attempt_no=0}) took 2214 ms. Trace:
I20260812 08:01:07.232717  7803 rpcz_store.cc:276] 0812 08:01:05.018458 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:05.018966 (+   508us) service_pool.cc:224] Handling call
0812 08:01:07.232669 (+2213703us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:05.019783 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:05.019851 (+    68us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:05.019854 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:05.019855 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:05.019934 (+    79us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:05.019939 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:05.019941 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:05.019999 (+    58us) tablet.cc:701] Row locks acquired
0812 08:01:05.020000 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:05.020027 (+    27us) write_op.cc:270] Start()
0812 08:01:05.020044 (+    17us) write_op.cc:276] Timestamp: P: 1786521665019991 usec, L: 0
0812 08:01:05.020046 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:05.020143 (+    97us) log.cc:844] Serialized 686 byte log entry
0812 08:01:07.232525 (+2212382us) op_driver.cc:464] REPLICATION: finished
0812 08:01:07.232531 (+     6us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 08:01:07.232534 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:07.232557 (+    23us) write_op.cc:454] Released schema lock
0812 08:01:07.232574 (+    17us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":557,"prepare.run_cpu_time_us":391,"prepare.run_wall_time_us":390,"replication_time_us":2212449}]]}
I20260812 08:01:07.232967  7803 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting uncommitted WRITE_OP operation due to leader change: 5.870
W20260812 08:01:07.233074  7803 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45238: Aborted: Op aborted by new leader
I20260812 08:01:07.233175  7803 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:45238 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=869, attempt_no=0}) took 2214 ms. Trace:
I20260812 08:01:07.233213  7803 rpcz_store.cc:276] 0812 08:01:05.019091 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:05.019248 (+   157us) service_pool.cc:224] Handling call
0812 08:01:07.233166 (+2213918us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:05.020213 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:05.020278 (+    65us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:05.020281 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:05.020282 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:05.020359 (+    77us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:05.020364 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:05.020367 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:05.020425 (+    58us) tablet.cc:701] Row locks acquired
0812 08:01:05.020426 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:05.020453 (+    27us) write_op.cc:270] Start()
0812 08:01:05.020470 (+    17us) write_op.cc:276] Timestamp: P: 1786521665020417 usec, L: 0
0812 08:01:05.020471 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:01:05.020569 (+    98us) log.cc:844] Serialized 696 byte log entry
0812 08:01:07.233037 (+2212468us) op_driver.cc:464] REPLICATION: finished
0812 08:01:07.233043 (+     6us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 08:01:07.233046 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:07.233063 (+    17us) write_op.cc:454] Released schema lock
0812 08:01:07.233069 (+     6us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":758,"prepare.run_cpu_time_us":389,"prepare.run_wall_time_us":388,"replication_time_us":2212535}]]}
I20260812 08:01:07.233440  7803 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting uncommitted WRITE_OP operation due to leader change: 5.871
W20260812 08:01:07.233552  7803 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45238: Aborted: Op aborted by new leader
I20260812 08:01:07.233667  7803 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:45238 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=867, attempt_no=0}) took 2214 ms. Trace:
I20260812 08:01:07.233702  7803 rpcz_store.cc:276] 0812 08:01:05.018952 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:05.019476 (+   524us) service_pool.cc:224] Handling call
0812 08:01:07.233657 (+2214181us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:05.020628 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:05.020692 (+    64us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:05.020695 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:05.020697 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:05.020773 (+    76us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:05.020778 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:05.020781 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:05.020839 (+    58us) tablet.cc:701] Row locks acquired
0812 08:01:05.020840 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:05.020866 (+    26us) write_op.cc:270] Start()
0812 08:01:05.020882 (+    16us) write_op.cc:276] Timestamp: P: 1786521665020830 usec, L: 0
0812 08:01:05.020884 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:05.021009 (+   125us) log.cc:844] Serialized 686 byte log entry
0812 08:01:07.233516 (+2212507us) op_driver.cc:464] REPLICATION: finished
0812 08:01:07.233521 (+     5us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 08:01:07.233524 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:07.233544 (+    20us) write_op.cc:454] Released schema lock
0812 08:01:07.233549 (+     5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"spinlock_wait_cycles":11520,"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":913,"prepare.run_cpu_time_us":413,"prepare.run_wall_time_us":412,"replication_time_us":2212599}]]}
I20260812 08:01:07.234030  7803 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting uncommitted WRITE_OP operation due to leader change: 5.872
W20260812 08:01:07.234146  7803 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45238: Aborted: Op aborted by new leader
I20260812 08:01:07.234241  7803 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:45238 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=870, attempt_no=0}) took 2212 ms. Trace:
I20260812 08:01:07.234277  7803 rpcz_store.cc:276] 0812 08:01:05.021961 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:05.022004 (+    43us) service_pool.cc:224] Handling call
0812 08:01:07.234234 (+2212230us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:05.025349 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:05.025434 (+    85us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:05.025437 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:05.025439 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:05.025522 (+    83us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:05.025527 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:05.025529 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:05.025575 (+    46us) tablet.cc:701] Row locks acquired
0812 08:01:05.025576 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:05.025604 (+    28us) write_op.cc:270] Start()
0812 08:01:05.025620 (+    16us) write_op.cc:276] Timestamp: P: 1786521665025567 usec, L: 0
0812 08:01:05.025622 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:05.025726 (+   104us) log.cc:844] Serialized 696 byte log entry
0812 08:01:07.234106 (+2208380us) op_driver.cc:464] REPLICATION: finished
0812 08:01:07.234111 (+     5us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 08:01:07.234115 (+     4us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:07.234137 (+    22us) write_op.cc:454] Released schema lock
0812 08:01:07.234143 (+     6us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":3070,"prepare.run_cpu_time_us":429,"prepare.run_wall_time_us":430,"replication_time_us":2208453}]]}
I20260812 08:01:07.235211  7803 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting uncommitted WRITE_OP operation due to leader change: 5.873
W20260812 08:01:07.235404  7803 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45238: Aborted: Op aborted by new leader
I20260812 08:01:07.235584  7803 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:45238 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=864, attempt_no=0}) took 2217 ms. Trace:
I20260812 08:01:07.235674  9144 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Connected to new peer: Peer: permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 867, Last known committed idx: 866, Time since last communication: 0.000s
I20260812 08:01:07.235682  7803 rpcz_store.cc:276] 0812 08:01:05.018339 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:05.018381 (+    42us) service_pool.cc:224] Handling call
0812 08:01:07.235573 (+2217192us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:05.025801 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:05.025873 (+    72us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:05.025876 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:05.025877 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:05.025959 (+    82us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:05.025964 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:05.025967 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:05.026026 (+    59us) tablet.cc:701] Row locks acquired
0812 08:01:05.026027 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:05.026052 (+    25us) write_op.cc:270] Start()
0812 08:01:05.026069 (+    17us) write_op.cc:276] Timestamp: P: 1786521665026017 usec, L: 0
0812 08:01:05.026070 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:01:05.026154 (+    84us) log.cc:844] Serialized 696 byte log entry
0812 08:01:07.235361 (+2209207us) op_driver.cc:464] REPLICATION: finished
0812 08:01:07.235367 (+     6us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 08:01:07.235370 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:07.235394 (+    24us) write_op.cc:454] Released schema lock
0812 08:01:07.235400 (+     6us) write_op.cc:335] FINISH: Op aborted
Metrics: {"spinlock_wait_cycles":13568,"child_traces":[["op",{"mutex_wait_us":3647,"num_ops":10,"prepare.queue_time_us":7103,"prepare.run_cpu_time_us":395,"prepare.run_wall_time_us":395,"replication_time_us":2209251}]]}
I20260812 08:01:07.236284  7803 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting uncommitted WRITE_OP operation due to leader change: 5.874
W20260812 08:01:07.236475  7803 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45238: Aborted: Op aborted by new leader
I20260812 08:01:07.236658  7803 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:45238 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=868, attempt_no=0}) took 2217 ms. Trace:
I20260812 08:01:07.236718  7803 rpcz_store.cc:276] 0812 08:01:05.019015 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:05.022374 (+  3359us) service_pool.cc:224] Handling call
0812 08:01:07.236648 (+2214274us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:05.026224 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:05.026293 (+    69us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:05.026295 (+     2us) write_op.cc:435] Acquired schema lock
0812 08:01:05.026297 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:05.026375 (+    78us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:05.026380 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:05.026383 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:05.026438 (+    55us) tablet.cc:701] Row locks acquired
0812 08:01:05.026439 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:05.026462 (+    23us) write_op.cc:270] Start()
0812 08:01:05.026477 (+    15us) write_op.cc:276] Timestamp: P: 1786521665026427 usec, L: 0
0812 08:01:05.026478 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:01:05.026560 (+    82us) log.cc:844] Serialized 696 byte log entry
0812 08:01:07.236431 (+2209871us) op_driver.cc:464] REPLICATION: finished
0812 08:01:07.236438 (+     7us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 08:01:07.236441 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:07.236463 (+    22us) write_op.cc:454] Released schema lock
0812 08:01:07.236469 (+     6us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":3646,"prepare.run_cpu_time_us":378,"prepare.run_wall_time_us":377,"replication_time_us":2209910}]]}
W20260812 08:01:07.237030  6905 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38762: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
I20260812 08:01:07.237699  9141 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 867, Last known committed idx: 866, Time since last communication: 0.000s
W20260812 08:01:07.239095  6905 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38762: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:07.239271  6899 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38762: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:07.240139  6899 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38762: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
I20260812 08:01:07.826759  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 7043
I20260812 08:01:07.849653  8004 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 6 FOLLOWER]: Refusing update from remote peer 56b60424ed9f4a079689ad82a318d29a: Log matching property violated. Preceding OpId in replica: term: 5 index: 788. Preceding OpId from leader: term: 6 index: 867. (index mismatch)
W20260812 08:01:07.846735  8305 meta_cache.cc:302] tablet f22cf4f6412248aa9e68c6d5d61153e8: replica 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363) has failed: Network error: recv got EOF from 127.5.142.130:38363 (error 108)
W20260812 08:01:07.854039  7905 connection.cc:466] server connection from 127.5.142.130:53931 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.130:53931 (request call id 3) could send its response
W20260812 08:01:07.854194  7287 connection.cc:466] server connection from 127.5.142.130:37813 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.130:37813 (request call id 12) could send its response
W20260812 08:01:07.858312  8118 connection.cc:466] server connection from 127.5.142.130:58147 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.130:58147 (request call id 33) could send its response
W20260812 08:01:07.858850  7715 connection.cc:466] server connection from 127.5.142.130:56785 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.130:56785 (request call id 64) could send its response
I20260812 08:01:09.010955  9186 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:09.011134  9186 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:09.011402  9187 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:09.011534  9187 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:09.012377  8968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 7 candidate_status { last_received { term: 6 index: 955 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:09.012399  6966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 7 candidate_status { last_received { term: 6 index: 955 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:09.012540  8968 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 6.
I20260812 08:01:09.012545  6966 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 6.
I20260812 08:01:09.012980  9186 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:09.013041  7803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 7 candidate_status { last_received { term: 6 index: 955 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:09.013542  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 7 candidate_status { last_received { term: 6 index: 955 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:09.013691  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 6.
I20260812 08:01:09.013846  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 7 candidate_status { last_received { term: 6 index: 955 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:09.013875  8226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 7 candidate_status { last_received { term: 6 index: 955 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:09.013936  7801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 7 candidate_status { last_received { term: 6 index: 955 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:09.014544  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 7 candidate_status { last_received { term: 6 index: 955 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:09.014614  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 7 candidate_status { last_received { term: 6 index: 955 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:09.014642  9187 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:09.014830  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 7 candidate_status { last_received { term: 6 index: 955 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:09.014938  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 6.
W20260812 08:01:09.015501  6868 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260812 08:01:09.015578  8872 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111)
W20260812 08:01:09.016301  6868 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363): Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111)
I20260812 08:01:09.016368  6868 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 3ae3f7a6e9f943e996a6dcea504878d0, 780d7e64a8214418a371f35351b113df, faafbc72cbd444ff90977a2cdb091b72; no voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6
I20260812 08:01:09.016534  9187 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260812 08:01:09.016785  8872 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363): Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111)
I20260812 08:01:09.016857  8872 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 3ae3f7a6e9f943e996a6dcea504878d0, 780d7e64a8214418a371f35351b113df, faafbc72cbd444ff90977a2cdb091b72; no voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6
I20260812 08:01:09.017020  9186 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 08:01:09.350382  9194 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:09.350544  9194 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:09.351336  9194 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:09.351677  8968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 7 candidate_status { last_received { term: 6 index: 877 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:09.351877  8968 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16901c5559a748a9aa770d73e254099c for term 7 because replica has last-logged OpId of term: 6 index: 955, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 877.
I20260812 08:01:09.351841  6966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 7 candidate_status { last_received { term: 6 index: 877 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:09.352031  6966 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16901c5559a748a9aa770d73e254099c for term 7 because replica has last-logged OpId of term: 6 index: 955, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 877.
I20260812 08:01:09.352002  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 7 candidate_status { last_received { term: 6 index: 877 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
W20260812 08:01:09.352234  7286 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111)
I20260812 08:01:09.352272  7801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 7 candidate_status { last_received { term: 6 index: 877 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:09.352640  7286 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 16901c5559a748a9aa770d73e254099c; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:09.352814  9194 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 08:01:09.352828  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 7 candidate_status { last_received { term: 6 index: 877 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:09.352993  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16901c5559a748a9aa770d73e254099c in term 6.
W20260812 08:01:09.353277  7286 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363): Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111)
I20260812 08:01:09.354637  9198 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:09.354766  9198 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:09.355427  9198 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:09.355841  8968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 908 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:09.355885  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 908 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:09.355988  7378 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6801d45ca6614156906e5d571ebcb8a6 in term 6.
I20260812 08:01:09.356011  8968 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6801d45ca6614156906e5d571ebcb8a6 for term 7 because replica has last-logged OpId of term: 6 index: 955, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 908.
I20260812 08:01:09.355998  6966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 908 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:09.356148  6966 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6801d45ca6614156906e5d571ebcb8a6 for term 7 because replica has last-logged OpId of term: 6 index: 955, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 908.
I20260812 08:01:09.356254  7801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 908 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
W20260812 08:01:09.356441  8116 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111)
I20260812 08:01:09.356808  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 908 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:09.356983  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6801d45ca6614156906e5d571ebcb8a6 in term 6.
W20260812 08:01:09.357368  8116 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363): Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111)
I20260812 08:01:09.357437  8116 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:09.357580  9198 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 08:01:09.358012  9201 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:09.358106  9201 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:09.358783  9201 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:09.359084  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 7 candidate_status { last_received { term: 6 index: 948 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:09.359203  8225 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 6.
I20260812 08:01:09.359230  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 7 candidate_status { last_received { term: 6 index: 948 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:09.359218  6966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 7 candidate_status { last_received { term: 6 index: 948 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:09.359218  8968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 7 candidate_status { last_received { term: 6 index: 948 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:09.359341  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 6.
I20260812 08:01:09.359367  6966 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1664e32d3cd0473592085aa522379f22 for term 7 because replica has last-logged OpId of term: 6 index: 955, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 948.
I20260812 08:01:09.359367  8968 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1664e32d3cd0473592085aa522379f22 for term 7 because replica has last-logged OpId of term: 6 index: 955, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 948.
I20260812 08:01:09.359805  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 7 candidate_status { last_received { term: 6 index: 948 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:09.359936  7378 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 6.
W20260812 08:01:09.359961  7713 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111) [suppressed 4 similar messages]
I20260812 08:01:09.360261  7715 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 3ae3f7a6e9f943e996a6dcea504878d0, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:09.360419  9201 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 08:01:09.360468  9201 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:09.360499  9201 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 6 FOLLOWER]: Advancing to term 7
W20260812 08:01:09.360693  7713 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363): Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111)
I20260812 08:01:09.361469  9201 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:09.362085  9201 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 7 election: Requested vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:09.362167  6966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 7 candidate_status { last_received { term: 6 index: 948 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0"
I20260812 08:01:09.362296  6966 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 6 FOLLOWER]: Advancing to term 7
I20260812 08:01:09.362411  8968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 7 candidate_status { last_received { term: 6 index: 948 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72"
I20260812 08:01:09.362493  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 7 candidate_status { last_received { term: 6 index: 948 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
I20260812 08:01:09.362525  8968 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 6 FOLLOWER]: Advancing to term 7
I20260812 08:01:09.362592  8225 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 6 FOLLOWER]: Advancing to term 7
I20260812 08:01:09.363092  6966 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1664e32d3cd0473592085aa522379f22 for term 7 because replica has last-logged OpId of term: 6 index: 955, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 948.
I20260812 08:01:09.363540  8225 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 7.
I20260812 08:01:09.363575  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 7 candidate_status { last_received { term: 6 index: 948 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:01:09.363595  8968 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1664e32d3cd0473592085aa522379f22 for term 7 because replica has last-logged OpId of term: 6 index: 955, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 948.
I20260812 08:01:09.363698  8004 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 6 FOLLOWER]: Advancing to term 7
I20260812 08:01:09.364338  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 7 candidate_status { last_received { term: 6 index: 948 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:01:09.364450  7378 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 6 FOLLOWER]: Advancing to term 7
W20260812 08:01:09.364446  7713 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363): Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111)
I20260812 08:01:09.364729  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 7.
I20260812 08:01:09.365476  7378 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 7.
I20260812 08:01:09.365742  7715 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:09.365922  9201 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 7 FOLLOWER]: Leader election won for term 7
I20260812 08:01:09.366101  9201 raft_consensus.cc:697] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 7 LEADER]: Becoming Leader. State: Replica: 1664e32d3cd0473592085aa522379f22, State: Running, Role: LEADER
I20260812 08:01:09.366276  9201 consensus_queue.cc:237] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 947, Committed index: 947, Last appended: 6.948, Last appended by leader: 948, Current term: 7, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:09.368494  6748 catalog_manager.cc:5719] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 reported cstate change: term changed from 6 to 7, leader changed from 56b60424ed9f4a079689ad82a318d29a (127.5.142.130) to 1664e32d3cd0473592085aa522379f22 (127.5.142.133). New cstate: current_term: 7 leader_uuid: "1664e32d3cd0473592085aa522379f22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } health_report { overall_health: UNKNOWN } } }
I20260812 08:01:09.401257  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.130:38363
--local_ip_for_outbound_sockets=127.5.142.130
--tserver_master_addrs=127.5.142.190:44379
--webserver_port=38317
--webserver_interface=127.5.142.130
--builtin_ntp_servers=127.5.142.148:43055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
W20260812 08:01:09.514786  9220 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:09.515015  9220 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:09.515058  9220 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:09.518451  9220 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:09.518556  9220 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.130
I20260812 08:01:09.522496  9220 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:43055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.130:38363
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.5.142.130
--webserver_port=38317
--tserver_master_addrs=127.5.142.190:44379
--never_fsync=true
--heap_profile_path=/tmp/kudu.9220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.130

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:09.523582  9220 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:09.524693  9220 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:09.531503  9229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:09.531623  9227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:09.531594  9220 server_base.cc:1061] running on GCE node
W20260812 08:01:09.531610  9231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:09.532141  9220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:09.532716  9220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:09.533931  9220 hybrid_clock.cc:648] HybridClock initialized: now 1786521669533906 us; error 37 us; skew 500 ppm
I20260812 08:01:09.535815  9220 webserver.cc:533] Webserver started at http://127.5.142.130:38317/ using document root <none> and password file <none>
I20260812 08:01:09.536415  9220 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:09.536499  9220 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:09.539990  9220 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:01:09.542007  9240 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:09.543042  9220 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:01:09.543154  9220 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "56b60424ed9f4a079689ad82a318d29a"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-ncp9"
I20260812 08:01:09.543545  9220 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:09.567591  9220 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:09.568315  9220 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:09.568499  9220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:09.569155  9220 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:09.570433  9251 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:01:09.573966  9220 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:01:09.574040  9220 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 08:01:09.574111  9220 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:01:09.575657  9220 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:01:09.575703  9220 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 08:01:09.576077  9251 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Bootstrap starting.
I20260812 08:01:09.586745  9220 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.130:38363
I20260812 08:01:09.586764  9420 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.130:38363 every 8 connection(s)
I20260812 08:01:09.587803  9220 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:01:09.588968  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 9220
I20260812 08:01:09.602465  9421 heartbeater.cc:344] Connected to a master server at 127.5.142.190:44379
I20260812 08:01:09.602789  9421 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:09.603474  9421 heartbeater.cc:507] Master 127.5.142.190:44379 requested a full tablet report, sending...
I20260812 08:01:09.605450  6748 ts_manager.cc:194] Re-registered known tserver with Master: 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363)
I20260812 08:01:09.606482  6748 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.130:56939
I20260812 08:01:09.640210  9251 log.cc:826] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:09.755218  8004 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 7 FOLLOWER]: Refusing update from remote peer 1664e32d3cd0473592085aa522379f22: Log matching property violated. Preceding OpId in replica: term: 5 index: 788. Preceding OpId from leader: term: 7 index: 949. (index mismatch)
I20260812 08:01:09.755810  9201 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 949, Last known committed idx: 788, Time since last communication: 0.000s
I20260812 08:01:09.759305  8968 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 7 FOLLOWER]: Refusing update from remote peer 1664e32d3cd0473592085aa522379f22: Log matching property violated. Preceding OpId in replica: term: 6 index: 955. Preceding OpId from leader: term: 7 index: 949. (term mismatch)
I20260812 08:01:09.759380  8968 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting all ops after (but not including) 948
I20260812 08:01:09.759404  8968 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 6.949
I20260812 08:01:09.759626  8968 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 6.950
I20260812 08:01:09.759785  8968 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 6.951
I20260812 08:01:09.759940  8968 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 6.952
I20260812 08:01:09.760084  8968 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 6.953
I20260812 08:01:09.760228  8968 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 6.954
I20260812 08:01:09.760368  8968 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 6.955
I20260812 08:01:09.760986  9201 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 949, Last known committed idx: 947, Time since last communication: 0.000s
I20260812 08:01:09.766258  6966 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 7 FOLLOWER]: Refusing update from remote peer 1664e32d3cd0473592085aa522379f22: Log matching property violated. Preceding OpId in replica: term: 6 index: 955. Preceding OpId from leader: term: 7 index: 949. (term mismatch)
I20260812 08:01:09.766353  6966 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting all ops after (but not including) 948
I20260812 08:01:09.766379  6966 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 6.949
I20260812 08:01:09.766662  6966 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38762 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=945, attempt_no=1}) took 1909 ms. Trace:
I20260812 08:01:09.766767  6966 rpcz_store.cc:276] 0812 08:01:07.857491 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:07.857549 (+    58us) service_pool.cc:224] Handling call
0812 08:01:09.766647 (+1909098us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 08:01:09.767012  6966 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 6.950
I20260812 08:01:09.767184  6966 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38762 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=944, attempt_no=1}) took 1909 ms. Trace:
I20260812 08:01:09.767251  6966 rpcz_store.cc:276] 0812 08:01:07.857575 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:07.857970 (+   395us) service_pool.cc:224] Handling call
0812 08:01:09.767173 (+1909203us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 08:01:09.767395  6966 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 6.951
I20260812 08:01:09.767532  6966 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38762 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=947, attempt_no=1}) took 1909 ms. Trace:
I20260812 08:01:09.767591  6966 rpcz_store.cc:276] 0812 08:01:07.857745 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:07.857769 (+    24us) service_pool.cc:224] Handling call
0812 08:01:09.767524 (+1909755us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 08:01:09.767742  6966 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 6.952
I20260812 08:01:09.767891  6966 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38762 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=946, attempt_no=1}) took 1910 ms. Trace:
I20260812 08:01:09.767946  6966 rpcz_store.cc:276] 0812 08:01:07.857812 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:07.857832 (+    20us) service_pool.cc:224] Handling call
0812 08:01:09.767882 (+1910050us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 08:01:09.768090  6966 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 6.953
I20260812 08:01:09.768244  6966 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38762 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=948, attempt_no=1}) took 1910 ms. Trace:
I20260812 08:01:09.768301  6966 rpcz_store.cc:276] 0812 08:01:07.857870 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:07.857890 (+    20us) service_pool.cc:224] Handling call
0812 08:01:09.768234 (+1910344us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 08:01:09.768447  6966 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 6.954
I20260812 08:01:09.768602  6966 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38762 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=949, attempt_no=1}) took 1910 ms. Trace:
I20260812 08:01:09.768661  6966 rpcz_store.cc:276] 0812 08:01:07.857690 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:07.857720 (+    30us) service_pool.cc:224] Handling call
0812 08:01:09.768592 (+1910872us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 08:01:09.768821  6966 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 6.955
I20260812 08:01:09.769002  6966 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38762 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=950, attempt_no=1}) took 1911 ms. Trace:
I20260812 08:01:09.769064  6966 rpcz_store.cc:276] 0812 08:01:07.857633 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:07.857674 (+    41us) service_pool.cc:224] Handling call
0812 08:01:09.768992 (+1911318us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 08:01:09.769693  9201 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 949, Last known committed idx: 947, Time since last communication: 0.000s
W20260812 08:01:09.773001  6905 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38762: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:09.774014  6905 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38762: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:09.776420  6899 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38762: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:09.777065  6905 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38762: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:09.778505  6893 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38762: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:09.778964  6899 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38762: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:09.779556  6905 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38762: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
I20260812 08:01:09.781621  7378 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 7 FOLLOWER]: Refusing update from remote peer 1664e32d3cd0473592085aa522379f22: Log matching property violated. Preceding OpId in replica: term: 6 index: 877. Preceding OpId from leader: term: 7 index: 950. (index mismatch)
I20260812 08:01:09.782151  9201 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 949, Last known committed idx: 876, Time since last communication: 0.000s
I20260812 08:01:09.789746  8225 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 7 FOLLOWER]: Refusing update from remote peer 1664e32d3cd0473592085aa522379f22: Log matching property violated. Preceding OpId in replica: term: 6 index: 908. Preceding OpId from leader: term: 7 index: 950. (index mismatch)
I20260812 08:01:09.792982  9205 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 949, Last known committed idx: 907, Time since last communication: 0.000s
W20260812 08:01:09.799435  7713 consensus_peers.cc:597] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 -> Peer 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363): Couldn't send request to peer 56b60424ed9f4a079689ad82a318d29a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:01:09.819128  9481 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38762 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=943, attempt_no=1}) took 1961 ms. Trace:
I20260812 08:01:09.819231  9481 rpcz_store.cc:276] 0812 08:01:07.857926 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:07.857946 (+    20us) service_pool.cc:224] Handling call
0812 08:01:09.819102 (+1961156us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260812 08:01:09.930138  8006 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.GetConsensusState from 127.0.0.1:53842 (request call id 103) took 81 ms (client timeout 100 ms). Trace:
W20260812 08:01:09.930245  8006 rpcz_store.cc:269] 0812 08:01:09.848256 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:09.849013 (+   757us) service_pool.cc:224] Handling call
0812 08:01:09.930027 (+ 81014us) spinlock_profiling.cc:243] Waited 80.7 ms on lock 0x56046d5ea554. stack: 00007f5e2170dd35 00007f5e2170e325 00007f5e2121a82d 000056043cdd2b18 000056043cdd2b6b 00007f5e2452edf1 00007f5e2452ed5f 00007f5e2376cbe1 00007f5e248565f1 00007f5e22eb7ac3 00007f5e22ebde1a 00007f5e2273cf8f 00007f5e2273c843 00007f5e2273f541 00007f5e2273de9e 00007f5e2273f8cb
0812 08:01:09.930115 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":185711488}
I20260812 08:01:10.676707  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 7687
W20260812 08:01:10.693703  8305 meta_cache.cc:302] tablet f22cf4f6412248aa9e68c6d5d61153e8: replica 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105) has failed: Network error: recv got EOF from 127.5.142.133:33105 (error 108)
W20260812 08:01:10.700351  8112 connection.cc:466] server connection from 127.5.142.133:60187 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.133:60187 (request call id 249) could send its response
W20260812 08:01:10.701957  7284 connection.cc:466] server connection from 127.5.142.133:48437 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.133:48437 (request call id 187) could send its response
I20260812 08:01:10.705870  9421 heartbeater.cc:499] Master 127.5.142.190:44379 was elected leader, sending a full tablet report...
W20260812 08:01:10.706822  7907 connection.cc:466] server connection from 127.5.142.133:57793 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.133:57793 (request call id 155) could send its response
W20260812 08:01:10.707093  9241 connection.cc:466] server connection from 127.5.142.133:39217 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.133:39217 (request call id 2) could send its response
I20260812 08:01:10.835932  9251 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Bootstrap replayed 1/1 log segments. Stats: ops{read=955 overwritten=0 applied=947 ignored=0} inserts{seen=9430 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 8 replicates
I20260812 08:01:10.836469  9251 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Bootstrap complete.
I20260812 08:01:10.837580  9251 ts_tablet_manager.cc:1403] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Time spent bootstrapping tablet: real 1.262s	user 0.743s	sys 0.055s
I20260812 08:01:10.839704  9251 raft_consensus.cc:359] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 6 FOLLOWER]: Replica starting. Triggering 8 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:10.841822  9251 raft_consensus.cc:740] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56b60424ed9f4a079689ad82a318d29a, State: Initialized, Role: FOLLOWER
I20260812 08:01:10.842335  9251 consensus_queue.cc:260] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 947, Last appended: 6.955, Last appended by leader: 955, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:10.843289  9251 ts_tablet_manager.cc:1434] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260812 08:01:11.632671  9521 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1664e32d3cd0473592085aa522379f22)
I20260812 08:01:11.632844  9521 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:11.633814  9521 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:11.634208  6968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 8 candidate_status { last_received { term: 7 index: 1003 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:11.634281  8962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 8 candidate_status { last_received { term: 7 index: 1003 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:11.634378  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 8 candidate_status { last_received { term: 7 index: 1003 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:11.635460  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 8 candidate_status { last_received { term: 7 index: 1003 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
W20260812 08:01:11.635640  7286 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105): Network error: Client connection negotiation failed: client connection to 127.5.142.133:33105: connect: Connection refused (error 111)
I20260812 08:01:11.635721  7286 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 16901c5559a748a9aa770d73e254099c; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 780d7e64a8214418a371f35351b113df, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:11.635885  9521 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 08:01:11.638302  9347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 8 candidate_status { last_received { term: 7 index: 1003 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:11.638612  9347 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16901c5559a748a9aa770d73e254099c in term 6.
I20260812 08:01:11.726502  9528 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1664e32d3cd0473592085aa522379f22)
I20260812 08:01:11.726647  9528 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:11.727363  9528 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:11.727813  6968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 1028 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:11.727813  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 1028 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:11.727963  7378 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6801d45ca6614156906e5d571ebcb8a6 in term 7.
I20260812 08:01:11.727959  8962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 1028 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:11.728470  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 1028 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
W20260812 08:01:11.729555  8116 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105): Network error: Client connection negotiation failed: client connection to 127.5.142.133:33105: connect: Connection refused (error 111)
I20260812 08:01:11.729631  8116 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 780d7e64a8214418a371f35351b113df, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:11.729777  9528 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 08:01:11.731799  9347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 1028 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:11.731923  9347 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6801d45ca6614156906e5d571ebcb8a6 in term 6.
I20260812 08:01:11.738247  9543 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1664e32d3cd0473592085aa522379f22)
I20260812 08:01:11.738366  9543 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:11.738647  9544 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1664e32d3cd0473592085aa522379f22)
I20260812 08:01:11.738749  9544 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:11.739050  9543 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:11.739306  8962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 8 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:11.739439  8962 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 7.
I20260812 08:01:11.739713  6968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 8 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:11.739756  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 8 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:11.739826  6968 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 7.
I20260812 08:01:11.739871  7378 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 7.
I20260812 08:01:11.740317  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 8 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:11.740921  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 8 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:11.741043  7378 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 7.
I20260812 08:01:11.741108  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 8 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:11.741230  8225 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 7.
I20260812 08:01:11.741349  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 8 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:11.741786  6867 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72; no voters: 780d7e64a8214418a371f35351b113df
I20260812 08:01:11.741962  9544 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:11.742107  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 8 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:11.742236  8225 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 7.
I20260812 08:01:11.742502  9543 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 08:01:11.742522  8873 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72; no voters: 780d7e64a8214418a371f35351b113df
I20260812 08:01:11.742571  9543 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1664e32d3cd0473592085aa522379f22)
I20260812 08:01:11.742614  9543 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 7 FOLLOWER]: Advancing to term 8
W20260812 08:01:11.742743  6868 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105): Network error: Client connection negotiation failed: client connection to 127.5.142.133:33105: connect: Connection refused (error 111)
I20260812 08:01:11.742769  9544 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 08:01:11.742818  9544 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1664e32d3cd0473592085aa522379f22)
I20260812 08:01:11.742853  9544 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 7 FOLLOWER]: Advancing to term 8
I20260812 08:01:11.743851  9543 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
W20260812 08:01:11.744050  8872 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105): Network error: Client connection negotiation failed: client connection to 127.5.142.133:33105: connect: Connection refused (error 111)
I20260812 08:01:11.743958  9544 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:11.744536  9543 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 8 election: Requested vote from peers faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:11.744976  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 8 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:01:11.745108  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 8 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
I20260812 08:01:11.745224  8225 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 7 FOLLOWER]: Advancing to term 8
W20260812 08:01:11.745447  6868 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105): Network error: Client connection negotiation failed: client connection to 127.5.142.133:33105: connect: Connection refused (error 111)
I20260812 08:01:11.746151  6968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 8 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0"
I20260812 08:01:11.746285  6968 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate faafbc72cbd444ff90977a2cdb091b72 in current term 8: Already voted for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in this term.
I20260812 08:01:11.746294  8225 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 8.
I20260812 08:01:11.746400  8962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 8 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72"
W20260812 08:01:11.746451  8872 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105): Network error: Client connection negotiation failed: client connection to 127.5.142.133:33105: connect: Connection refused (error 111)
I20260812 08:01:11.746547  8962 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ae3f7a6e9f943e996a6dcea504878d0 in current term 8: Already voted for candidate faafbc72cbd444ff90977a2cdb091b72 in this term.
I20260812 08:01:11.746621  9347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 8 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:11.746701  9346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 8 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a"
I20260812 08:01:11.746763  9347 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 6.
I20260812 08:01:11.746884  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 8 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:01:11.746992  7378 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 7 FOLLOWER]: Advancing to term 8
I20260812 08:01:11.747545  6868 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 3ae3f7a6e9f943e996a6dcea504878d0; no voters: 1664e32d3cd0473592085aa522379f22, 56b60424ed9f4a079689ad82a318d29a, 780d7e64a8214418a371f35351b113df, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:11.747857  7375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 8 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:01:11.748102  7378 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 8.
I20260812 08:01:11.748373  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 8 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:01:11.748531  9543 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260812 08:01:11.748735  8871 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: faafbc72cbd444ff90977a2cdb091b72; no voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, 780d7e64a8214418a371f35351b113df
I20260812 08:01:11.749073  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 8 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
I20260812 08:01:11.749229  8225 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate faafbc72cbd444ff90977a2cdb091b72 in current term 8: Already voted for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in this term.
I20260812 08:01:11.749301  9544 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 8 election: Requested vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:11.749312  9554 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260812 08:01:11.750443  9346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 8 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:11.750504  9347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 8 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a"
I20260812 08:01:11.750571  9346 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 6.
I20260812 08:01:12.196517  9556 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1664e32d3cd0473592085aa522379f22)
I20260812 08:01:12.196674  9556 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:12.197536  9556 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183)
I20260812 08:01:12.197791  8962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 8 candidate_status { last_received { term: 7 index: 980 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:12.197821  6968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 8 candidate_status { last_received { term: 7 index: 980 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:12.197984  8962 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df in current term 8: Already voted for candidate faafbc72cbd444ff90977a2cdb091b72 in this term.
I20260812 08:01:12.197986  6968 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df in current term 8: Already voted for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in this term.
W20260812 08:01:12.198457  7902 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.142.133:33105: connect: Connection refused (error 111)
I20260812 08:01:12.198573  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 8 candidate_status { last_received { term: 7 index: 980 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:12.198655  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 8 candidate_status { last_received { term: 7 index: 980 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:12.198735  7378 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df in current term 8: Already voted for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in this term.
I20260812 08:01:12.198817  8225 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df in current term 8: Already voted for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in this term.
I20260812 08:01:12.199090  7905 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 780d7e64a8214418a371f35351b113df; no voters: 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:12.199306  9556 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 7 FOLLOWER]: Advancing to term 8
W20260812 08:01:12.199465  7902 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105): Network error: Client connection negotiation failed: client connection to 127.5.142.133:33105: connect: Connection refused (error 111)
I20260812 08:01:12.200266  9556 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 08:01:12.202556  9347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 8 candidate_status { last_received { term: 7 index: 980 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:12.202677  9347 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 780d7e64a8214418a371f35351b113df in term 6.
I20260812 08:01:13.586419  9569 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:01:13.586577  9569 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:13.587289  9569 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:13.587608  6968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 9 candidate_status { last_received { term: 7 index: 1003 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:13.587795  6968 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16901c5559a748a9aa770d73e254099c for term 9 because replica has last-logged OpId of term: 7 index: 1033, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 1003.
I20260812 08:01:13.587934  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 9 candidate_status { last_received { term: 7 index: 1003 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:13.588102  8225 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16901c5559a748a9aa770d73e254099c for term 9 because replica has last-logged OpId of term: 7 index: 1028, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 1003.
I20260812 08:01:13.588161  8962 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 9 candidate_status { last_received { term: 7 index: 1003 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:13.588238  9347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 9 candidate_status { last_received { term: 7 index: 1003 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:13.588348  8962 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16901c5559a748a9aa770d73e254099c for term 9 because replica has last-logged OpId of term: 7 index: 1033, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 1003.
I20260812 08:01:13.588373  9347 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16901c5559a748a9aa770d73e254099c in term 6.
I20260812 08:01:13.588670  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 9 candidate_status { last_received { term: 7 index: 1003 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
W20260812 08:01:13.588851  7286 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105): Network error: Client connection negotiation failed: client connection to 127.5.142.133:33105: connect: Connection refused (error 111)
I20260812 08:01:13.588857  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16901c5559a748a9aa770d73e254099c in term 8.
I20260812 08:01:13.588968  7286 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 56b60424ed9f4a079689ad82a318d29a; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:13.589121  9569 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260812 08:01:13.591862  9575 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:01:13.591984  9575 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:13.592949  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 9 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:13.592969  6968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 9 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:13.593004  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 9 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:13.593084  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 8.
I20260812 08:01:13.593123  6968 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 8.
I20260812 08:01:13.593130  8225 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 8.
I20260812 08:01:13.593137  9347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 9 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:13.593257  9347 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 6.
I20260812 08:01:13.593509  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 9 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:13.593660  7378 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 8.
I20260812 08:01:13.593580  8871 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 3ae3f7a6e9f943e996a6dcea504878d0, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df, faafbc72cbd444ff90977a2cdb091b72; no voters: 
I20260812 08:01:13.592845  9575 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:13.594038  9578 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260812 08:01:13.594106  9578 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:01:13.594153  9578 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 8 FOLLOWER]: Advancing to term 9
W20260812 08:01:13.594637  8872 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105): Network error: Client connection negotiation failed: client connection to 127.5.142.133:33105: connect: Connection refused (error 111)
I20260812 08:01:13.595148  9578 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:13.595862  6968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 9 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0"
I20260812 08:01:13.595991  6968 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 8 FOLLOWER]: Advancing to term 9
I20260812 08:01:13.595980  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 9 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:01:13.595981  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 9 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
I20260812 08:01:13.596083  7378 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 8 FOLLOWER]: Advancing to term 9
I20260812 08:01:13.596093  8225 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 8 FOLLOWER]: Advancing to term 9
W20260812 08:01:13.596832  8872 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105): Network error: Client connection negotiation failed: client connection to 127.5.142.133:33105: connect: Connection refused (error 111)
I20260812 08:01:13.597044  7378 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 9.
I20260812 08:01:13.597044  8225 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 9.
I20260812 08:01:13.597163  6968 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 9.
I20260812 08:01:13.597503  9578 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 9 election: Requested vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:13.597503  8875 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72; no voters: 1664e32d3cd0473592085aa522379f22
I20260812 08:01:13.597680  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 9 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:01:13.597679  9347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 9 candidate_status { last_received { term: 7 index: 1033 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a"
I20260812 08:01:13.597782  9578 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 9 FOLLOWER]: Leader election won for term 9
I20260812 08:01:13.597800  8004 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 8 FOLLOWER]: Advancing to term 9
I20260812 08:01:13.597806  9347 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 6 FOLLOWER]: Advancing to term 9
I20260812 08:01:13.597877  9578 raft_consensus.cc:697] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 9 LEADER]: Becoming Leader. State: Replica: faafbc72cbd444ff90977a2cdb091b72, State: Running, Role: LEADER
I20260812 08:01:13.598187  9578 consensus_queue.cc:237] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1025, Committed index: 1025, Last appended: 7.1033, Last appended by leader: 1033, Current term: 9, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:13.598819  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 9.
I20260812 08:01:13.599793  9347 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 9.
I20260812 08:01:13.601421  6748 catalog_manager.cc:5719] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 reported cstate change: term changed from 7 to 9, leader changed from 1664e32d3cd0473592085aa522379f22 (127.5.142.133) to faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132). New cstate: current_term: 9 leader_uuid: "faafbc72cbd444ff90977a2cdb091b72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } health_report { overall_health: UNKNOWN } } }
I20260812 08:01:13.987252  9347 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 9 FOLLOWER]: Refusing update from remote peer faafbc72cbd444ff90977a2cdb091b72: Log matching property violated. Preceding OpId in replica: term: 6 index: 955. Preceding OpId from leader: term: 9 index: 1034. (index mismatch)
I20260812 08:01:13.987879  9575 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1034, Last known committed idx: 947, Time since last communication: 0.000s
I20260812 08:01:13.988036  9575 consensus_queue.cc:1243] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Peer 56b60424ed9f4a079689ad82a318d29a log is divergent from this leader: its last log entry 6.955 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 947
I20260812 08:01:13.990636  9347 raft_consensus.cc:1217] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 9 FOLLOWER]: Deduplicated request from leader. Original: 6.947->[6.948-9.1034]   Dedup: 6.948->[7.949-9.1034]
I20260812 08:01:13.990826  9347 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting all ops after (but not including) 948
I20260812 08:01:13.990876  9347 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting uncommitted WRITE_OP operation due to leader change: 6.949
I20260812 08:01:13.991374  9347 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting uncommitted WRITE_OP operation due to leader change: 6.950
I20260812 08:01:13.991556  9347 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting uncommitted WRITE_OP operation due to leader change: 6.951
I20260812 08:01:13.991714  9347 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting uncommitted WRITE_OP operation due to leader change: 6.952
I20260812 08:01:13.991860  9347 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting uncommitted WRITE_OP operation due to leader change: 6.953
I20260812 08:01:13.992012  9347 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting uncommitted WRITE_OP operation due to leader change: 6.954
I20260812 08:01:13.992144  9347 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting uncommitted WRITE_OP operation due to leader change: 6.955
I20260812 08:01:14.023604  9595 mvcc.cc:204] Tried to move back new op lower bound from 7317592761182076928 to 7317592757726920704. Current Snapshot: MvccSnapshot[applied={T|T < 7317592761182076928 or (T in {7317592761182076928})}]
I20260812 08:01:14.027304  7378 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 9 FOLLOWER]: Refusing update from remote peer faafbc72cbd444ff90977a2cdb091b72: Log matching property violated. Preceding OpId in replica: term: 7 index: 1003. Preceding OpId from leader: term: 9 index: 1034. (index mismatch)
I20260812 08:01:14.027788  9575 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1034, Last known committed idx: 1002, Time since last communication: 0.000s
I20260812 08:01:14.091727  6968 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 9 FOLLOWER]: Refusing update from remote peer faafbc72cbd444ff90977a2cdb091b72: Log matching property violated. Preceding OpId in replica: term: 7 index: 1033. Preceding OpId from leader: term: 9 index: 1034. (index mismatch)
I20260812 08:01:14.092361  9575 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1034, Last known committed idx: 1025, Time since last communication: 0.000s
W20260812 08:01:14.097728  8872 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.142.133:33105: connect: Connection refused (error 111) [suppressed 4 similar messages]
I20260812 08:01:14.097764  8004 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 9 FOLLOWER]: Refusing update from remote peer faafbc72cbd444ff90977a2cdb091b72: Log matching property violated. Preceding OpId in replica: term: 7 index: 980. Preceding OpId from leader: term: 9 index: 1034. (index mismatch)
I20260812 08:01:14.098220  8225 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 9 FOLLOWER]: Refusing update from remote peer faafbc72cbd444ff90977a2cdb091b72: Log matching property violated. Preceding OpId in replica: term: 7 index: 1028. Preceding OpId from leader: term: 9 index: 1034. (index mismatch)
I20260812 08:01:14.100411  9623 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38762 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1020, attempt_no=1}) took 3389 ms. Trace:
I20260812 08:01:14.101750  9623 rpcz_store.cc:276] 0812 08:01:10.710892 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:10.711272 (+   380us) service_pool.cc:224] Handling call
0812 08:01:14.100381 (+3389109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:01:14.101013  9628 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38762 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1022, attempt_no=1}) took 3390 ms. Trace:
I20260812 08:01:14.102218  9628 rpcz_store.cc:276] 0812 08:01:10.711002 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:10.711060 (+    58us) service_pool.cc:224] Handling call
0812 08:01:14.100981 (+3389921us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:01:14.102949  9628 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38762 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1025, attempt_no=1}) took 3391 ms. Trace:
I20260812 08:01:14.103019  9628 rpcz_store.cc:276] 0812 08:01:10.711186 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:10.711211 (+    25us) service_pool.cc:224] Handling call
0812 08:01:14.102937 (+3391726us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:01:14.103060  9627 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38762 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1024, attempt_no=1}) took 3392 ms. Trace:
I20260812 08:01:14.103144  9627 rpcz_store.cc:276] 0812 08:01:10.710800 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:10.710847 (+    47us) service_pool.cc:224] Handling call
0812 08:01:14.103029 (+3392182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:01:14.103750  9628 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38762 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1026, attempt_no=1}) took 3392 ms. Trace:
I20260812 08:01:14.103839  9628 rpcz_store.cc:276] 0812 08:01:10.711083 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:10.711116 (+    33us) service_pool.cc:224] Handling call
0812 08:01:14.103736 (+3392620us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:01:14.103894  9627 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38762 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1027, attempt_no=1}) took 3392 ms. Trace:
I20260812 08:01:14.103986  9627 rpcz_store.cc:276] 0812 08:01:10.711138 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:10.711376 (+   238us) service_pool.cc:224] Handling call
0812 08:01:14.103882 (+3392506us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:01:14.101073  9624 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38762 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1021, attempt_no=1}) took 3390 ms. Trace:
I20260812 08:01:14.104228  9624 rpcz_store.cc:276] 0812 08:01:10.710942 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:10.710987 (+    45us) service_pool.cc:224] Handling call
0812 08:01:14.101041 (+3390054us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:01:14.104826  9623 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:38762 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1023, attempt_no=1}) took 3393 ms. Trace:
I20260812 08:01:14.104966  9623 rpcz_store.cc:276] 0812 08:01:10.711242 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:10.711272 (+    30us) service_pool.cc:224] Handling call
0812 08:01:14.104813 (+3393541us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260812 08:01:14.106192  8872 consensus_peers.cc:597] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 -> Peer 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105): Couldn't send request to peer 1664e32d3cd0473592085aa522379f22. Status: Network error: Client connection negotiation failed: client connection to 127.5.142.133:33105: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260812 08:01:14.107087  6904 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38762: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:14.107673  6899 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38762: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:14.107082  6893 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38762: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
I20260812 08:01:14.108099  9630 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1034, Last known committed idx: 979, Time since last communication: 0.000s
W20260812 08:01:14.108497  6904 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38762: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:14.109032  6904 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38762: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
I20260812 08:01:14.109086  9630 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1034, Last known committed idx: 1025, Time since last communication: 0.000s
W20260812 08:01:14.109259  6905 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38762: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:14.109755  6893 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38762: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:14.110280  6899 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38762: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:14.119638  8157 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39074: Illegal state: replica 6801d45ca6614156906e5d571ebcb8a6 is not leader of this config: current role FOLLOWER
W20260812 08:01:14.119964  8149 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39074: Illegal state: replica 6801d45ca6614156906e5d571ebcb8a6 is not leader of this config: current role FOLLOWER
W20260812 08:01:14.120016  8141 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39074: Illegal state: replica 6801d45ca6614156906e5d571ebcb8a6 is not leader of this config: current role FOLLOWER
W20260812 08:01:14.119625  8158 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39074: Illegal state: replica 6801d45ca6614156906e5d571ebcb8a6 is not leader of this config: current role FOLLOWER
W20260812 08:01:14.120525  8145 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39074: Illegal state: replica 6801d45ca6614156906e5d571ebcb8a6 is not leader of this config: current role FOLLOWER
W20260812 08:01:14.119618  8155 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39074: Illegal state: replica 6801d45ca6614156906e5d571ebcb8a6 is not leader of this config: current role FOLLOWER
W20260812 08:01:14.121248  8147 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39074: Illegal state: replica 6801d45ca6614156906e5d571ebcb8a6 is not leader of this config: current role FOLLOWER
W20260812 08:01:14.121385  8146 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39074: Illegal state: replica 6801d45ca6614156906e5d571ebcb8a6 is not leader of this config: current role FOLLOWER
W20260812 08:01:14.123811  7317 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:40918: Illegal state: replica 16901c5559a748a9aa770d73e254099c is not leader of this config: current role FOLLOWER
W20260812 08:01:14.123811  7321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:40918: Illegal state: replica 16901c5559a748a9aa770d73e254099c is not leader of this config: current role FOLLOWER
W20260812 08:01:14.124818  7317 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:40918: Illegal state: replica 16901c5559a748a9aa770d73e254099c is not leader of this config: current role FOLLOWER
W20260812 08:01:14.126155  7321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:40918: Illegal state: replica 16901c5559a748a9aa770d73e254099c is not leader of this config: current role FOLLOWER
W20260812 08:01:14.126844  7317 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:40918: Illegal state: replica 16901c5559a748a9aa770d73e254099c is not leader of this config: current role FOLLOWER
W20260812 08:01:14.127249  7321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:40918: Illegal state: replica 16901c5559a748a9aa770d73e254099c is not leader of this config: current role FOLLOWER
W20260812 08:01:14.142735  7943 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35090: Illegal state: replica 780d7e64a8214418a371f35351b113df is not leader of this config: current role FOLLOWER
W20260812 08:01:14.142737  7941 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35090: Illegal state: replica 780d7e64a8214418a371f35351b113df is not leader of this config: current role FOLLOWER
W20260812 08:01:14.143014  7940 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35090: Illegal state: replica 780d7e64a8214418a371f35351b113df is not leader of this config: current role FOLLOWER
W20260812 08:01:14.142756  7942 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35090: Illegal state: replica 780d7e64a8214418a371f35351b113df is not leader of this config: current role FOLLOWER
W20260812 08:01:14.143334  7936 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35090: Illegal state: replica 780d7e64a8214418a371f35351b113df is not leader of this config: current role FOLLOWER
W20260812 08:01:14.143656  7938 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35090: Illegal state: replica 780d7e64a8214418a371f35351b113df is not leader of this config: current role FOLLOWER
W20260812 08:01:14.144127  7939 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35090: Illegal state: replica 780d7e64a8214418a371f35351b113df is not leader of this config: current role FOLLOWER
W20260812 08:01:14.144488  7935 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35090: Illegal state: replica 780d7e64a8214418a371f35351b113df is not leader of this config: current role FOLLOWER
W20260812 08:01:14.147746  7321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:40918: Illegal state: replica 16901c5559a748a9aa770d73e254099c is not leader of this config: current role FOLLOWER
I20260812 08:01:14.385700  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.133:33105
--local_ip_for_outbound_sockets=127.5.142.133
--tserver_master_addrs=127.5.142.190:44379
--webserver_port=36547
--webserver_interface=127.5.142.133
--builtin_ntp_servers=127.5.142.148:43055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
W20260812 08:01:14.735935  9688 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:14.736251  9688 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:14.736348  9688 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:14.740685  9688 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:14.740833  9688 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.133
I20260812 08:01:14.746446  9688 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:43055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.133:33105
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.5.142.133
--webserver_port=36547
--tserver_master_addrs=127.5.142.190:44379
--never_fsync=true
--heap_profile_path=/tmp/kudu.9688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.133

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:14.748059  9688 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:14.750253  9688 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:14.777308  9696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:14.777508  9699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:14.777971  9695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:14.785745  9688 server_base.cc:1061] running on GCE node
I20260812 08:01:14.787135  9688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:14.787995  9688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:14.854897  9688 hybrid_clock.cc:648] HybridClock initialized: now 1786521674854804 us; error 126 us; skew 500 ppm
I20260812 08:01:14.857692  9688 webserver.cc:533] Webserver started at http://127.5.142.133:36547/ using document root <none> and password file <none>
I20260812 08:01:14.858534  9688 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:14.858639  9688 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:14.886828  9688 fs_manager.cc:714] Time spent opening directory manager: real 0.027s	user 0.002s	sys 0.003s
I20260812 08:01:14.895190  9705 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:14.896668  9688 fs_manager.cc:730] Time spent opening block manager: real 0.008s	user 0.002s	sys 0.001s
I20260812 08:01:14.896878  9688 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "1664e32d3cd0473592085aa522379f22"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-ncp9"
I20260812 08:01:14.898038  9688 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:14.953483  9688 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:14.954461  9688 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:14.954726  9688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:14.956072  9688 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:14.972373  9716 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:01:14.999886  9688 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:01:14.999975  9688 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.042s	user 0.001s	sys 0.000s
I20260812 08:01:15.000049  9688 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:01:15.002285  9688 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:01:15.002336  9688 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:01:15.020123  9716 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Bootstrap starting.
I20260812 08:01:15.046895  9688 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.133:33105
I20260812 08:01:15.049855  9688 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20260812 08:01:15.054937  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 9688
I20260812 08:01:15.061467  9885 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.133:33105 every 8 connection(s)
I20260812 08:01:15.126127  9886 heartbeater.cc:344] Connected to a master server at 127.5.142.190:44379
I20260812 08:01:15.126405  9886 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:15.127121  9886 heartbeater.cc:507] Master 127.5.142.190:44379 requested a full tablet report, sending...
I20260812 08:01:15.129173  6746 ts_manager.cc:194] Re-registered known tserver with Master: 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105)
I20260812 08:01:15.130141  6746 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.133:43737
I20260812 08:01:15.211139  9716 log.cc:826] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:16.134097  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 8841
W20260812 08:01:16.151819  8304 meta_cache.cc:302] tablet f22cf4f6412248aa9e68c6d5d61153e8: replica faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153) has failed: Network error: recv got EOF from 127.5.142.132:43153 (error 108)
W20260812 08:01:16.157130  7898 connection.cc:466] server connection from 127.5.142.132:60243 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.132:60243 (request call id 255) could send its response
I20260812 08:01:16.159687  9886 heartbeater.cc:499] Master 127.5.142.190:44379 was elected leader, sending a full tablet report...
W20260812 08:01:16.162585  8116 connection.cc:466] server connection from 127.5.142.132:33571 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.132:33571 (request call id 300) could send its response
W20260812 08:01:16.166783  7287 connection.cc:466] server connection from 127.5.142.132:38275 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.132:38275 (request call id 219) could send its response
I20260812 08:01:16.331947  9716 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Bootstrap replayed 1/1 log segments. Stats: ops{read=1041 overwritten=8 applied=1025 ignored=0} inserts{seen=10200 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 8 replicates
I20260812 08:01:16.332477  9716 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Bootstrap complete.
I20260812 08:01:16.333614  9716 ts_tablet_manager.cc:1403] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Time spent bootstrapping tablet: real 1.314s	user 0.865s	sys 0.051s
I20260812 08:01:16.335654  9716 raft_consensus.cc:359] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 7 FOLLOWER]: Replica starting. Triggering 8 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:16.337682  9716 raft_consensus.cc:740] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1664e32d3cd0473592085aa522379f22, State: Initialized, Role: FOLLOWER
I20260812 08:01:16.338165  9716 consensus_queue.cc:260] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1025, Last appended: 7.1033, Last appended by leader: 1033, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:16.338982  9716 ts_tablet_manager.cc:1434] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.004s
I20260812 08:01:16.969455  9904 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 9 FOLLOWER]: Starting pre-election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:01:16.969624  9904 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:16.970966  9904 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183)
I20260812 08:01:16.971000  6966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 10 candidate_status { last_received { term: 9 index: 1337 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:16.971288  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 10 candidate_status { last_received { term: 9 index: 1337 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:16.971639  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 10 candidate_status { last_received { term: 9 index: 1337 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:16.972018  9347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 10 candidate_status { last_received { term: 9 index: 1337 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:16.972466  7902 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 780d7e64a8214418a371f35351b113df; no voters: 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6
I20260812 08:01:16.972651  9904 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260812 08:01:16.973044  7898 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153): Network error: Client connection negotiation failed: client connection to 127.5.142.132:43153: connect: Connection refused (error 111)
I20260812 08:01:16.975088  9809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 10 candidate_status { last_received { term: 9 index: 1337 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:16.975394  9809 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 780d7e64a8214418a371f35351b113df in term 7.
I20260812 08:01:17.205847  9911 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:01:17.205901  9912 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:01:17.206032  9911 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:17.206087  9912 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:17.206976  9911 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:17.207810  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 10 candidate_status { last_received { term: 9 index: 1407 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:17.207839  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 10 candidate_status { last_received { term: 9 index: 1407 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:17.207996  8008 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 9.
I20260812 08:01:17.208020  9347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 10 candidate_status { last_received { term: 9 index: 1407 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:17.208141  9347 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 9.
W20260812 08:01:17.208221  6867 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.142.132:43153: connect: Connection refused (error 111) [suppressed 2 similar messages]
I20260812 08:01:17.208405  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 10 candidate_status { last_received { term: 9 index: 1407 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
W20260812 08:01:17.209697  6867 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153): Network error: Client connection negotiation failed: client connection to 127.5.142.132:43153: connect: Connection refused (error 111)
W20260812 08:01:17.212319  9241 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.142.132:43153: connect: Connection refused (error 111)
I20260812 08:01:17.212739  9912 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:17.212837  9809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 10 candidate_status { last_received { term: 9 index: 1407 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:17.213006  9809 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 7.
I20260812 08:01:17.213615  6868 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, 780d7e64a8214418a371f35351b113df; no voters: 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:17.213907  9911 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260812 08:01:17.213958  9911 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 9 FOLLOWER]: Starting leader election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:01:17.213994  9911 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 9 FOLLOWER]: Advancing to term 10
I20260812 08:01:17.215173  9911 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:17.215860  9911 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 10 election: Requested vote from peers faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:17.216367  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 10 candidate_status { last_received { term: 9 index: 1407 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:01:17.216404  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 10 candidate_status { last_received { term: 9 index: 1407 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:01:17.216513  8008 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 9 FOLLOWER]: Advancing to term 10
I20260812 08:01:17.217011  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 10 candidate_status { last_received { term: 9 index: 1407 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
W20260812 08:01:17.217150  6867 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153): Network error: Client connection negotiation failed: client connection to 127.5.142.132:43153: connect: Connection refused (error 111)
I20260812 08:01:17.217351  9347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 10 candidate_status { last_received { term: 9 index: 1407 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a"
I20260812 08:01:17.217487  9347 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 9 FOLLOWER]: Advancing to term 10
I20260812 08:01:17.217478  9809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 10 candidate_status { last_received { term: 9 index: 1407 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22"
I20260812 08:01:17.217618  9809 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 7 FOLLOWER]: Advancing to term 10
I20260812 08:01:17.219517  9347 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 10.
I20260812 08:01:17.219689  8008 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 10.
W20260812 08:01:17.220065  9241 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153): Network error: Client connection negotiation failed: client connection to 127.5.142.132:43153: connect: Connection refused (error 111)
I20260812 08:01:17.220237  9810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 10 candidate_status { last_received { term: 9 index: 1407 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:17.220656  6966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 10 candidate_status { last_received { term: 9 index: 1407 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:17.220801  6966 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56b60424ed9f4a079689ad82a318d29a in current term 10: Already voted for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in this term.
I20260812 08:01:17.221706  9809 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 10.
I20260812 08:01:17.222064  6868 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, 780d7e64a8214418a371f35351b113df; no voters: 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:17.223052  9911 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 10 FOLLOWER]: Leader election won for term 10
I20260812 08:01:17.223026  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 10 candidate_status { last_received { term: 9 index: 1407 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:17.223189  8008 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56b60424ed9f4a079689ad82a318d29a in current term 10: Already voted for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in this term.
I20260812 08:01:17.223292  9911 raft_consensus.cc:697] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 10 LEADER]: Becoming Leader. State: Replica: 3ae3f7a6e9f943e996a6dcea504878d0, State: Running, Role: LEADER
I20260812 08:01:17.223471  9911 consensus_queue.cc:237] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1399, Committed index: 1399, Last appended: 9.1407, Last appended by leader: 1407, Current term: 10, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:17.223515  9241 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 56b60424ed9f4a079689ad82a318d29a; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 780d7e64a8214418a371f35351b113df, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:17.223807  9912 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260812 08:01:17.225301  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 10 candidate_status { last_received { term: 9 index: 1407 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:17.226656  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.132:43153
--local_ip_for_outbound_sockets=127.5.142.132
--tserver_master_addrs=127.5.142.190:44379
--webserver_port=43191
--webserver_interface=127.5.142.132
--builtin_ntp_servers=127.5.142.148:43055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
I20260812 08:01:17.226646  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 10 candidate_status { last_received { term: 9 index: 1407 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:17.226577  6746 catalog_manager.cc:5719] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 reported cstate change: term changed from 9 to 10, leader changed from faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132) to 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129). New cstate: current_term: 10 leader_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } health_report { overall_health: UNKNOWN } } }
W20260812 08:01:17.344463  9944 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:17.344764  9944 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:17.344818  9944 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:17.348379  9944 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:17.348518  9944 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.132
I20260812 08:01:17.352761  9944 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:43055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.132:43153
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.5.142.132
--webserver_port=43191
--tserver_master_addrs=127.5.142.190:44379
--never_fsync=true
--heap_profile_path=/tmp/kudu.9944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.132

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:17.354139  9944 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:17.355396  9944 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:17.362294  9953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:17.362355  9952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:17.362455  9944 server_base.cc:1061] running on GCE node
W20260812 08:01:17.362318  9957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:17.363013  9944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:17.363583  9944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:17.364745  9944 hybrid_clock.cc:648] HybridClock initialized: now 1786521677364715 us; error 51 us; skew 500 ppm
I20260812 08:01:17.366544  9944 webserver.cc:533] Webserver started at http://127.5.142.132:43191/ using document root <none> and password file <none>
I20260812 08:01:17.367163  9944 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:17.367220  9944 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:17.370687  9944 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:01:17.372609  9963 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:17.373631  9944 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:01:17.373731  9944 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "faafbc72cbd444ff90977a2cdb091b72"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-ncp9"
I20260812 08:01:17.374130  9944 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:17.402594  9944 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:17.403321  9944 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:17.403502  9944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:17.404116  9944 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:17.405470  9972 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:01:17.409039  9944 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:01:17.409106  9944 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 08:01:17.409160  9944 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:01:17.410692  9944 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:01:17.410735  9944 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 08:01:17.411098  9972 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Bootstrap starting.
I20260812 08:01:17.421517  9944 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.132:43153
I20260812 08:01:17.421543 10159 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.132:43153 every 8 connection(s)
I20260812 08:01:17.422600  9944 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20260812 08:01:17.425316  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 9944
I20260812 08:01:17.432035 10162 heartbeater.cc:344] Connected to a master server at 127.5.142.190:44379
I20260812 08:01:17.432322 10162 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:17.432981 10162 heartbeater.cc:507] Master 127.5.142.190:44379 requested a full tablet report, sending...
I20260812 08:01:17.434892  6746 ts_manager.cc:194] Re-registered known tserver with Master: faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153)
I20260812 08:01:17.435849  6746 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.132:33383
I20260812 08:01:17.545218  9972 log.cc:826] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:17.658684 10171 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:01:17.658836 10171 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:17.660171 10171 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:17.660838  9347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 1399 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:17.661126  9347 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6801d45ca6614156906e5d571ebcb8a6 in current term 10: Already voted for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in this term.
I20260812 08:01:17.660812  6966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 1399 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:17.665306  9809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 1399 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:17.665494  9809 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6801d45ca6614156906e5d571ebcb8a6 in current term 10: Already voted for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in this term.
I20260812 08:01:17.664788 10082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 1399 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
W20260812 08:01:17.666436  8112 leader_election.cc:343] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 10 pre-election: Tablet error from VoteRequest() call to peer faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:01:17.666520  8112 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 6801d45ca6614156906e5d571ebcb8a6; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:17.666698 10171 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 9 FOLLOWER]: Advancing to term 10
I20260812 08:01:17.667654 10171 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260812 08:01:17.759111  9347 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 10 FOLLOWER]: Refusing update from remote peer 3ae3f7a6e9f943e996a6dcea504878d0: Log matching property violated. Preceding OpId in replica: term: 9 index: 1407. Preceding OpId from leader: term: 10 index: 1408. (index mismatch)
I20260812 08:01:17.759222  9809 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 10 FOLLOWER]: Refusing update from remote peer 3ae3f7a6e9f943e996a6dcea504878d0: Log matching property violated. Preceding OpId in replica: term: 7 index: 1033. Preceding OpId from leader: term: 10 index: 1408. (index mismatch)
I20260812 08:01:17.759692 10174 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1408, Last known committed idx: 1399, Time since last communication: 0.000s
I20260812 08:01:17.759897  9911 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1408, Last known committed idx: 1025, Time since last communication: 0.000s
W20260812 08:01:17.824393  6867 consensus_peers.cc:597] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 -> Peer faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153): Couldn't send request to peer faafbc72cbd444ff90977a2cdb091b72. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:01:17.884356 10188 mvcc.cc:204] Tried to move back new op lower bound from 7317592783605362688 to 7317592775063072768. Current Snapshot: MvccSnapshot[applied={T|T < 7317592779812278272 or (T in {7317592779826155520,7317592779829178368,7317592779844886528,7317592779852021760})}]
I20260812 08:01:18.168475  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 6842
W20260812 08:01:18.190313  8309 meta_cache.cc:302] tablet f22cf4f6412248aa9e68c6d5d61153e8: replica 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863) has failed: Network error: recv got EOF from 127.5.142.129:41863 (error 108)
I20260812 08:01:18.192131  8004 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 10 FOLLOWER]: Refusing update from remote peer 3ae3f7a6e9f943e996a6dcea504878d0: Log matching property violated. Preceding OpId in replica: term: 9 index: 1337. Preceding OpId from leader: term: 10 index: 1408. (index mismatch)
I20260812 08:01:18.192471  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 1399 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:18.192620  8008 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6801d45ca6614156906e5d571ebcb8a6 in current term 10: Already voted for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in this term.
I20260812 08:01:18.195158  8225 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 10 FOLLOWER]: Refusing update from remote peer 3ae3f7a6e9f943e996a6dcea504878d0: Log matching property violated. Preceding OpId in replica: term: 9 index: 1399. Preceding OpId from leader: term: 10 index: 1408. (index mismatch)
I20260812 08:01:18.196539  7375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 1399 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:18.192540 10191 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:01:18.198894 10191 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:18.199079  7378 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 9 FOLLOWER]: Advancing to term 10
I20260812 08:01:18.200150  7378 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 10 FOLLOWER]: Refusing update from remote peer 3ae3f7a6e9f943e996a6dcea504878d0: Log matching property violated. Preceding OpId in replica: term: 9 index: 1363. Preceding OpId from leader: term: 10 index: 1408. (index mismatch)
I20260812 08:01:18.201236 10191 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:18.202018  9347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 10 candidate_status { last_received { term: 9 index: 1363 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:18.202852  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 10 candidate_status { last_received { term: 9 index: 1363 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:18.203015  8008 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16901c5559a748a9aa770d73e254099c in current term 10: Already voted for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in this term.
W20260812 08:01:18.203341  7905 connection.cc:466] server connection from 127.5.142.129:58217 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.129:58217 (request call id 11) could send its response
W20260812 08:01:18.204972  8118 connection.cc:466] server connection from 127.5.142.129:47269 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.129:47269 (request call id 11) could send its response
I20260812 08:01:18.205560  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 10 candidate_status { last_received { term: 9 index: 1363 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:18.205736  8225 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16901c5559a748a9aa770d73e254099c for term 10 because replica has last-logged OpId of term: 9 index: 1399, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 1363.
W20260812 08:01:18.207129  7288 connection.cc:466] server connection from 127.5.142.129:49475 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.129:49475 (request call id 11) could send its response
W20260812 08:01:18.212370  7288 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.142.129:41863: connect: Connection refused (error 111) [suppressed 2 similar messages]
I20260812 08:01:18.216655 10082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 10 candidate_status { last_received { term: 9 index: 1363 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
W20260812 08:01:18.217247  7284 leader_election.cc:343] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 10 pre-election: Tablet error from VoteRequest() call to peer faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:01:18.217377  7284 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 16901c5559a748a9aa770d73e254099c; no voters: 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:18.217540 10191 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260812 08:01:18.219031  7288 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863): Network error: Client connection negotiation failed: client connection to 127.5.142.129:41863: connect: Connection refused (error 111)
I20260812 08:01:18.221776  9809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 10 candidate_status { last_received { term: 9 index: 1363 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:18.437997 10162 heartbeater.cc:499] Master 127.5.142.190:44379 was elected leader, sending a full tablet report...
I20260812 08:01:18.661610  9972 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Bootstrap replayed 1/1 log segments. Stats: ops{read=1419 overwritten=12 applied=1399 ignored=0} inserts{seen=13930 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 8 replicates
I20260812 08:01:18.662261  9972 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Bootstrap complete.
I20260812 08:01:18.663348  9972 ts_tablet_manager.cc:1403] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Time spent bootstrapping tablet: real 1.253s	user 1.063s	sys 0.040s
I20260812 08:01:18.665017  9972 raft_consensus.cc:359] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 9 FOLLOWER]: Replica starting. Triggering 8 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:18.667138  9972 raft_consensus.cc:740] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 9 FOLLOWER]: Becoming Follower/Learner. State: Replica: faafbc72cbd444ff90977a2cdb091b72, State: Initialized, Role: FOLLOWER
I20260812 08:01:18.667615  9972 consensus_queue.cc:260] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1399, Last appended: 9.1407, Last appended by leader: 1407, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:18.668454  9972 ts_tablet_manager.cc:1434] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.004s
I20260812 08:01:18.857918 10210 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:01:18.858112 10210 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:18.858973 10210 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183)
I20260812 08:01:18.859524  9347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 11 candidate_status { last_received { term: 9 index: 1337 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:18.859581  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 11 candidate_status { last_received { term: 9 index: 1337 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:18.859772  8225 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df for term 11 because replica has last-logged OpId of term: 9 index: 1399, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 1337.
I20260812 08:01:18.859767  9809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 11 candidate_status { last_received { term: 9 index: 1337 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:18.860006  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 11 candidate_status { last_received { term: 9 index: 1337 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
W20260812 08:01:18.860076  7907 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.142.129:41863: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260812 08:01:18.860193  7378 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df for term 11 because replica has last-logged OpId of term: 9 index: 1363, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 1337.
I20260812 08:01:18.860526  7907 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6
I20260812 08:01:18.860742 10210 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260812 08:01:18.861114  7907 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863): Network error: Client connection negotiation failed: client connection to 127.5.142.129:41863: connect: Connection refused (error 111)
I20260812 08:01:18.863517 10082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 11 candidate_status { last_received { term: 9 index: 1337 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:18.863765 10082 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df for term 11 because replica has last-logged OpId of term: 9 index: 1407, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 1337.
I20260812 08:01:19.262943 10216 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3ae3f7a6e9f943e996a6dcea504878d0)
I20260812 08:01:19.263099 10216 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:19.264072 10216 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:19.264173  9809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 11 candidate_status { last_received { term: 10 index: 1408 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:19.264384  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 11 candidate_status { last_received { term: 10 index: 1408 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:19.264545  8225 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 10.
I20260812 08:01:19.264642  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 11 candidate_status { last_received { term: 10 index: 1408 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:19.264777  7378 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 10.
I20260812 08:01:19.265040  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 11 candidate_status { last_received { term: 10 index: 1408 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:19.265199  8008 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 10.
I20260812 08:01:19.265501  9241 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22
I20260812 08:01:19.265793 10216 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260812 08:01:19.265820  9246 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863): Network error: Client connection negotiation failed: client connection to 127.5.142.129:41863: connect: Connection refused (error 111)
I20260812 08:01:19.265892 10216 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3ae3f7a6e9f943e996a6dcea504878d0)
I20260812 08:01:19.265942 10216 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 10 FOLLOWER]: Advancing to term 11
I20260812 08:01:19.266902 10216 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:19.267817  9809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 11 candidate_status { last_received { term: 10 index: 1408 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22"
I20260812 08:01:19.268536  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 11 candidate_status { last_received { term: 10 index: 1408 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:01:19.268548  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 11 candidate_status { last_received { term: 10 index: 1408 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
W20260812 08:01:19.268564  9246 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863): Network error: Client connection negotiation failed: client connection to 127.5.142.129:41863: connect: Connection refused (error 111)
I20260812 08:01:19.268661  7378 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 10 FOLLOWER]: Advancing to term 11
I20260812 08:01:19.268666  8225 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 10 FOLLOWER]: Advancing to term 11
I20260812 08:01:19.268730 10082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 11 candidate_status { last_received { term: 10 index: 1408 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:19.268953 10082 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 9.
I20260812 08:01:19.269186 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 11 candidate_status { last_received { term: 10 index: 1408 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72"
I20260812 08:01:19.269287 10079 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 9 FOLLOWER]: Advancing to term 11
I20260812 08:01:19.269691  8225 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 11.
I20260812 08:01:19.269691  7378 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 11.
I20260812 08:01:19.269960 10216 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 11 election: Requested vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:19.270525  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 11 candidate_status { last_received { term: 10 index: 1408 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:01:19.270646  8008 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 10 FOLLOWER]: Advancing to term 11
I20260812 08:01:19.271148 10079 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 11.
I20260812 08:01:19.271379  8008 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 11.
I20260812 08:01:19.271431  9241 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0
I20260812 08:01:19.271616 10216 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 11 FOLLOWER]: Leader election won for term 11
I20260812 08:01:19.271836 10216 raft_consensus.cc:697] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 11 LEADER]: Becoming Leader. State: Replica: 56b60424ed9f4a079689ad82a318d29a, State: Running, Role: LEADER
I20260812 08:01:19.272142 10216 consensus_queue.cc:237] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1399, Committed index: 1399, Last appended: 10.1408, Last appended by leader: 1408, Current term: 11, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:19.273569 10226 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3ae3f7a6e9f943e996a6dcea504878d0)
I20260812 08:01:19.273677 10226 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:19.274919  6745 catalog_manager.cc:5719] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a reported cstate change: term changed from 10 to 11, leader changed from 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129) to 56b60424ed9f4a079689ad82a318d29a (127.5.142.130). New cstate: current_term: 11 leader_uuid: "56b60424ed9f4a079689ad82a318d29a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } health_report { overall_health: UNKNOWN } } }
W20260812 08:01:19.275775  9711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.142.129:41863: connect: Connection refused (error 111)
I20260812 08:01:19.276672 10226 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
W20260812 08:01:19.279230  9711 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863): Network error: Client connection negotiation failed: client connection to 127.5.142.129:41863: connect: Connection refused (error 111)
I20260812 08:01:19.280371  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 11 candidate_status { last_received { term: 10 index: 1408 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:19.280540  8225 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1664e32d3cd0473592085aa522379f22 in current term 11: Already voted for candidate 56b60424ed9f4a079689ad82a318d29a in this term.
I20260812 08:01:19.281260 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 11 candidate_status { last_received { term: 10 index: 1408 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:19.281452 10079 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1664e32d3cd0473592085aa522379f22 in current term 11: Already voted for candidate 56b60424ed9f4a079689ad82a318d29a in this term.
I20260812 08:01:19.282430  9347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 11 candidate_status { last_received { term: 10 index: 1408 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:19.282871  9707 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 1664e32d3cd0473592085aa522379f22; no voters: 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:19.283123 10226 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 10 FOLLOWER]: Advancing to term 11
I20260812 08:01:19.283629  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 11 candidate_status { last_received { term: 10 index: 1408 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:19.283785  7378 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1664e32d3cd0473592085aa522379f22 in current term 11: Already voted for candidate 56b60424ed9f4a079689ad82a318d29a in this term.
I20260812 08:01:19.284266 10226 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260812 08:01:19.284476  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 11 candidate_status { last_received { term: 10 index: 1408 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:19.284613  8008 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1664e32d3cd0473592085aa522379f22 in current term 11: Already voted for candidate 56b60424ed9f4a079689ad82a318d29a in this term.
I20260812 08:01:19.650756  9809 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 11 FOLLOWER]: Refusing update from remote peer 56b60424ed9f4a079689ad82a318d29a: Log matching property violated. Preceding OpId in replica: term: 10 index: 1408. Preceding OpId from leader: term: 11 index: 1409. (index mismatch)
I20260812 08:01:19.651374 10216 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1409, Last known committed idx: 1399, Time since last communication: 0.000s
I20260812 08:01:19.746918  7378 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 11 FOLLOWER]: Refusing update from remote peer 56b60424ed9f4a079689ad82a318d29a: Log matching property violated. Preceding OpId in replica: term: 9 index: 1363. Preceding OpId from leader: term: 11 index: 1409. (index mismatch)
I20260812 08:01:19.747409 10216 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Connected to new peer: Peer: permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1409, Last known committed idx: 1361, Time since last communication: 0.000s
I20260812 08:01:19.757025  8008 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 11 FOLLOWER]: Refusing update from remote peer 56b60424ed9f4a079689ad82a318d29a: Log matching property violated. Preceding OpId in replica: term: 9 index: 1337. Preceding OpId from leader: term: 11 index: 1409. (index mismatch)
I20260812 08:01:19.757750 10222 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Connected to new peer: Peer: permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1409, Last known committed idx: 1336, Time since last communication: 0.000s
I20260812 08:01:19.761286  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.129:41863
--local_ip_for_outbound_sockets=127.5.142.129
--tserver_master_addrs=127.5.142.190:44379
--webserver_port=33453
--webserver_interface=127.5.142.129
--builtin_ntp_servers=127.5.142.148:43055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
I20260812 08:01:19.780339  8225 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 11 FOLLOWER]: Refusing update from remote peer 56b60424ed9f4a079689ad82a318d29a: Log matching property violated. Preceding OpId in replica: term: 9 index: 1399. Preceding OpId from leader: term: 11 index: 1409. (index mismatch)
I20260812 08:01:19.782760 10222 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1409, Last known committed idx: 1398, Time since last communication: 0.000s
I20260812 08:01:19.784027 10079 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 11 FOLLOWER]: Refusing update from remote peer 56b60424ed9f4a079689ad82a318d29a: Log matching property violated. Preceding OpId in replica: term: 9 index: 1407. Preceding OpId from leader: term: 11 index: 1409. (index mismatch)
I20260812 08:01:19.788766 10302 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48992 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1393, attempt_no=2}) took 1575 ms. Trace:
I20260812 08:01:19.788954 10302 rpcz_store.cc:276] 0812 08:01:18.213259 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:18.214377 (+  1118us) service_pool.cc:224] Handling call
0812 08:01:19.788746 (+1574369us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:01:19.789829 10222 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Connected to new peer: Peer: permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1409, Last known committed idx: 1399, Time since last communication: 0.000s
W20260812 08:01:19.792044  9741 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48992: Illegal state: replica 1664e32d3cd0473592085aa522379f22 is not leader of this config: current role FOLLOWER
I20260812 08:01:19.795677 10302 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48992 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1394, attempt_no=2}) took 1582 ms. Trace:
I20260812 08:01:19.795779 10302 rpcz_store.cc:276] 0812 08:01:18.213527 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:18.214753 (+  1226us) service_pool.cc:224] Handling call
0812 08:01:19.795667 (+1580914us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:01:19.797564 10302 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48992 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1395, attempt_no=2}) took 1584 ms. Trace:
I20260812 08:01:19.797677 10302 rpcz_store.cc:276] 0812 08:01:18.213482 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:18.214697 (+  1215us) service_pool.cc:224] Handling call
0812 08:01:19.797553 (+1582856us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:01:19.799950 10301 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48992 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1397, attempt_no=2}) took 1586 ms. Trace:
I20260812 08:01:19.800082 10301 rpcz_store.cc:276] 0812 08:01:18.213352 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:18.214538 (+  1186us) service_pool.cc:224] Handling call
0812 08:01:19.799938 (+1585400us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:01:19.801679 10301 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48992 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1398, attempt_no=2}) took 1588 ms. Trace:
I20260812 08:01:19.801774 10301 rpcz_store.cc:276] 0812 08:01:18.213437 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:18.214636 (+  1199us) service_pool.cc:224] Handling call
0812 08:01:19.801669 (+1587033us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:01:19.803462 10301 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48992 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1399, attempt_no=2}) took 1590 ms. Trace:
I20260812 08:01:19.803566 10301 rpcz_store.cc:276] 0812 08:01:18.213170 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:18.213568 (+   398us) service_pool.cc:224] Handling call
0812 08:01:19.803451 (+1589883us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:01:19.804759 10301 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48992 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1400, attempt_no=2}) took 1591 ms. Trace:
I20260812 08:01:19.804852 10301 rpcz_store.cc:276] 0812 08:01:18.213395 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:18.214586 (+  1191us) service_pool.cc:224] Handling call
0812 08:01:19.804749 (+1590163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:01:19.817062 10302 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48992 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1396, attempt_no=2}) took 1603 ms. Trace:
I20260812 08:01:19.817842 10302 rpcz_store.cc:276] 0812 08:01:18.213308 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:18.214459 (+  1151us) service_pool.cc:224] Handling call
0812 08:01:19.817037 (+1602578us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260812 08:01:19.826054  9246 consensus_peers.cc:597] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a -> Peer 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863): Couldn't send request to peer 3ae3f7a6e9f943e996a6dcea504878d0. Status: Network error: Client connection negotiation failed: client connection to 127.5.142.129:41863: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260812 08:01:19.931769 10270 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:19.932190 10270 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:19.932324 10270 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:19.936924 10270 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:19.937096 10270 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.129
I20260812 08:01:19.943112 10270 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:43055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.129:41863
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.5.142.129
--webserver_port=33453
--tserver_master_addrs=127.5.142.190:44379
--never_fsync=true
--heap_profile_path=/tmp/kudu.10270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.129

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:19.945191 10270 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:19.947055 10270 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:19.964037 10350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:19.964811 10270 server_base.cc:1061] running on GCE node
W20260812 08:01:19.972522 10351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:19.972759 10355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:19.973376 10270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:19.974177 10270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:20.005920 10270 hybrid_clock.cc:648] HybridClock initialized: now 1786521679998726 us; error 7195 us; skew 500 ppm
I20260812 08:01:20.009246 10270 webserver.cc:533] Webserver started at http://127.5.142.129:33453/ using document root <none> and password file <none>
I20260812 08:01:20.010196 10270 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:20.010360 10270 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:20.015981 10270 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260812 08:01:20.019337 10362 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:20.021854 10270 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.001s	sys 0.000s
I20260812 08:01:20.022030 10270 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "3ae3f7a6e9f943e996a6dcea504878d0"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-ncp9"
I20260812 08:01:20.022614 10270 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:20.056285 10270 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:20.057255 10270 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:20.057540 10270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:20.058336 10270 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:20.062533 10380 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:01:20.075435 10270 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:01:20.075619 10270 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.016s	user 0.001s	sys 0.000s
I20260812 08:01:20.075724 10270 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:01:20.078011 10270 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:01:20.078085 10270 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:01:20.085387 10380 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Bootstrap starting.
I20260812 08:01:20.103138 10270 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.129:41863
I20260812 08:01:20.121515 10270 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:01:20.127863  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 10270
I20260812 08:01:20.162504 10552 heartbeater.cc:344] Connected to a master server at 127.5.142.190:44379
I20260812 08:01:20.162827 10552 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:20.163599 10552 heartbeater.cc:507] Master 127.5.142.190:44379 requested a full tablet report, sending...
I20260812 08:01:20.166465  6745 ts_manager.cc:194] Re-registered known tserver with Master: 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863)
I20260812 08:01:20.167564  6745 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.129:58925
I20260812 08:01:20.218991 10550 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.129:41863 every 8 connection(s)
I20260812 08:01:20.270924 10380 log.cc:826] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:21.245787  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 9220
W20260812 08:01:21.262140  8305 meta_cache.cc:302] tablet f22cf4f6412248aa9e68c6d5d61153e8: replica 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363) has failed: Network error: recv got EOF from 127.5.142.130:38363 (error 108)
W20260812 08:01:21.272572  7902 connection.cc:466] server connection from 127.5.142.130:56725 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.130:56725 (request call id 109) could send its response
W20260812 08:01:21.273869  7286 connection.cc:466] server connection from 127.5.142.130:39027 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.130:39027 (request call id 159) could send its response
I20260812 08:01:21.279670 10552 heartbeater.cc:499] Master 127.5.142.190:44379 was elected leader, sending a full tablet report...
W20260812 08:01:21.288359 10364 connection.cc:466] server connection from 127.5.142.130:48085 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.130:48085 (request call id 2) could send its response
W20260812 08:01:21.294523  8112 connection.cc:466] server connection from 127.5.142.130:45845 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.130:45845 (request call id 109) could send its response
W20260812 08:01:21.300755  9969 connection.cc:466] server connection from 127.5.142.130:60049 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.130:60049 (request call id 151) could send its response
I20260812 08:01:21.506337 10380 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Bootstrap replayed 1/1 log segments. Stats: ops{read=1428 overwritten=20 applied=1399 ignored=0} inserts{seen=13930 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 9 replicates
I20260812 08:01:21.506831 10380 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Bootstrap complete.
I20260812 08:01:21.507902 10380 ts_tablet_manager.cc:1403] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Time spent bootstrapping tablet: real 1.423s	user 1.143s	sys 0.048s
I20260812 08:01:21.510522 10380 raft_consensus.cc:359] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 10 FOLLOWER]: Replica starting. Triggering 9 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:21.513332 10380 raft_consensus.cc:740] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 10 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae3f7a6e9f943e996a6dcea504878d0, State: Initialized, Role: FOLLOWER
I20260812 08:01:21.513926 10380 consensus_queue.cc:260] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1399, Last appended: 10.1408, Last appended by leader: 1408, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:21.514901 10380 ts_tablet_manager.cc:1434] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Time spent starting tablet: real 0.007s	user 0.009s	sys 0.000s
I20260812 08:01:22.078186 10564 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:22.078336 10564 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:22.079092 10564 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183)
I20260812 08:01:22.079547 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 12 candidate_status { last_received { term: 11 index: 1564 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:22.079857  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 12 candidate_status { last_received { term: 11 index: 1564 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:22.079934  8224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 12 candidate_status { last_received { term: 11 index: 1564 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:22.080353  9809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 12 candidate_status { last_received { term: 11 index: 1564 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:22.080869  7902 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:22.081100 10564 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260812 08:01:22.081552  7902 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363): Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111)
I20260812 08:01:22.083657 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 12 candidate_status { last_received { term: 11 index: 1564 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:22.083958 10460 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 780d7e64a8214418a371f35351b113df in term 10.
I20260812 08:01:22.187019 10573 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:22.187206 10573 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:22.188081 10573 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:22.188768  9809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 12 candidate_status { last_received { term: 11 index: 1590 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:22.189026 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 12 candidate_status { last_received { term: 11 index: 1590 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:22.189075  8224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 12 candidate_status { last_received { term: 11 index: 1590 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:22.189590  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 12 candidate_status { last_received { term: 11 index: 1590 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:22.189745  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16901c5559a748a9aa770d73e254099c in term 11.
W20260812 08:01:22.190176  7286 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363): Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111)
I20260812 08:01:22.190263  7286 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:22.190418 10573 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260812 08:01:22.192538 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 12 candidate_status { last_received { term: 11 index: 1590 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:22.192673 10460 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16901c5559a748a9aa770d73e254099c in term 10.
I20260812 08:01:22.301954 10577 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:22.302100 10577 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:22.302817 10577 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:22.303352  9809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 1622 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:22.303452 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 1622 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:22.303563  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 1622 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:22.303735  7381 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6801d45ca6614156906e5d571ebcb8a6 in term 11.
I20260812 08:01:22.304057  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 1622 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:22.304219  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6801d45ca6614156906e5d571ebcb8a6 in term 11.
W20260812 08:01:22.304415  8116 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260812 08:01:22.305248  8116 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363): Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111)
I20260812 08:01:22.306895 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 1622 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:22.307029 10460 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6801d45ca6614156906e5d571ebcb8a6 in term 10.
I20260812 08:01:22.307296  8118 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 56b60424ed9f4a079689ad82a318d29a, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:22.307478 10577 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260812 08:01:22.307556 10577 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:22.307603 10577 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 11 FOLLOWER]: Advancing to term 12
I20260812 08:01:22.308441 10577 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:22.309048 10577 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 12 election: Requested vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:22.309257 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 1622 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72"
I20260812 08:01:22.309540  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 1622 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:01:22.309634  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 1622 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:01:22.309670  7381 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 11 FOLLOWER]: Advancing to term 12
I20260812 08:01:22.309767  8004 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 11 FOLLOWER]: Advancing to term 12
W20260812 08:01:22.310292  8116 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363): Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111)
I20260812 08:01:22.310622  7381 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6801d45ca6614156906e5d571ebcb8a6 in term 12.
I20260812 08:01:22.310760  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6801d45ca6614156906e5d571ebcb8a6 in term 12.
I20260812 08:01:22.311036  9809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 1622 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22"
I20260812 08:01:22.311604 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 1622 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0"
I20260812 08:01:22.311736 10460 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 10 FOLLOWER]: Advancing to term 12
I20260812 08:01:22.313478 10460 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6801d45ca6614156906e5d571ebcb8a6 in term 12.
I20260812 08:01:22.313781 10584 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:22.313815  8118 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 56b60424ed9f4a079689ad82a318d29a, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:22.313894 10584 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:22.313995 10577 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 12 FOLLOWER]: Leader election won for term 12
I20260812 08:01:22.314270 10577 raft_consensus.cc:697] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 12 LEADER]: Becoming Leader. State: Replica: 6801d45ca6614156906e5d571ebcb8a6, State: Running, Role: LEADER
I20260812 08:01:22.314625 10577 consensus_queue.cc:237] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1619, Committed index: 1619, Last appended: 11.1622, Last appended by leader: 1621, Current term: 12, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:22.315092 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 12 candidate_status { last_received { term: 11 index: 1627 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:22.315603 10584 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:22.315963  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 12 candidate_status { last_received { term: 11 index: 1627 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:22.316118  7381 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1664e32d3cd0473592085aa522379f22 in current term 12: Already voted for candidate 6801d45ca6614156906e5d571ebcb8a6 in this term.
I20260812 08:01:22.316195  8224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 12 candidate_status { last_received { term: 11 index: 1627 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:22.316378  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 12 candidate_status { last_received { term: 11 index: 1627 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:22.316531  8004 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1664e32d3cd0473592085aa522379f22 in current term 12: Already voted for candidate 6801d45ca6614156906e5d571ebcb8a6 in this term.
I20260812 08:01:22.316852  9706 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 1664e32d3cd0473592085aa522379f22; no voters: 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:22.317353 10584 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 11 FOLLOWER]: Advancing to term 12
I20260812 08:01:22.318039  6745 catalog_manager.cc:5719] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 reported cstate change: term changed from 11 to 12, leader changed from 56b60424ed9f4a079689ad82a318d29a (127.5.142.130) to 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135). New cstate: current_term: 12 leader_uuid: "6801d45ca6614156906e5d571ebcb8a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } health_report { overall_health: UNKNOWN } } }
W20260812 08:01:22.318266  9707 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363): Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111)
I20260812 08:01:22.318449 10584 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260812 08:01:22.319612 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 12 candidate_status { last_received { term: 11 index: 1627 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:22.319803 10460 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1664e32d3cd0473592085aa522379f22 in current term 12: Already voted for candidate 6801d45ca6614156906e5d571ebcb8a6 in this term.
I20260812 08:01:22.334998  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.130:38363
--local_ip_for_outbound_sockets=127.5.142.130
--tserver_master_addrs=127.5.142.190:44379
--webserver_port=38317
--webserver_interface=127.5.142.130
--builtin_ntp_servers=127.5.142.148:43055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
W20260812 08:01:22.446465 10598 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:22.446731 10598 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:22.446802 10598 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:22.450194 10598 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:22.450336 10598 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.130
I20260812 08:01:22.454275 10598 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:43055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.130:38363
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.5.142.130
--webserver_port=38317
--tserver_master_addrs=127.5.142.190:44379
--never_fsync=true
--heap_profile_path=/tmp/kudu.10598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.130

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:22.455406 10598 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:22.456581 10598 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:22.463029 10609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:22.462991 10611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:22.463042 10608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:22.463443 10598 server_base.cc:1061] running on GCE node
I20260812 08:01:22.463781 10598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:22.464335 10598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:22.465468 10598 hybrid_clock.cc:648] HybridClock initialized: now 1786521682465444 us; error 38 us; skew 500 ppm
I20260812 08:01:22.467104 10598 webserver.cc:533] Webserver started at http://127.5.142.130:38317/ using document root <none> and password file <none>
I20260812 08:01:22.467674 10598 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:22.467730 10598 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:22.471145 10598 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:01:22.473031 10620 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:22.474202 10598 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 08:01:22.474303 10598 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "56b60424ed9f4a079689ad82a318d29a"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-ncp9"
I20260812 08:01:22.474675 10598 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:22.489496 10598 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:22.490113 10598 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:22.490272 10598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:22.490826 10598 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:22.492075 10635 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:01:22.495491 10598 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:01:22.495563 10598 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 08:01:22.495631 10598 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:01:22.497175 10598 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:01:22.497229 10598 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:01:22.497551 10635 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Bootstrap starting.
I20260812 08:01:22.508071 10598 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.130:38363
I20260812 08:01:22.508116 10819 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.130:38363 every 8 connection(s)
I20260812 08:01:22.509172 10598 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:01:22.512516  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 10598
I20260812 08:01:22.520358 10820 heartbeater.cc:344] Connected to a master server at 127.5.142.190:44379
I20260812 08:01:22.520632 10820 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:22.521250 10820 heartbeater.cc:507] Master 127.5.142.190:44379 requested a full tablet report, sending...
I20260812 08:01:22.523264  6745 ts_manager.cc:194] Re-registered known tserver with Master: 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363)
I20260812 08:01:22.524169  6745 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.130:36667
I20260812 08:01:22.607095 10635 log.cc:826] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:22.719017  7381 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 12 FOLLOWER]: Refusing update from remote peer 6801d45ca6614156906e5d571ebcb8a6: Log matching property violated. Preceding OpId in replica: term: 11 index: 1590. Preceding OpId from leader: term: 12 index: 1623. (index mismatch)
I20260812 08:01:22.719599 10577 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1623, Last known committed idx: 1585, Time since last communication: 0.000s
I20260812 08:01:22.742234 10079 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 11 FOLLOWER]: Advancing to term 12
I20260812 08:01:22.743597 10079 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 12 FOLLOWER]: Refusing update from remote peer 6801d45ca6614156906e5d571ebcb8a6: Log matching property violated. Preceding OpId in replica: term: 11 index: 1627. Preceding OpId from leader: term: 12 index: 1623. (term mismatch)
I20260812 08:01:22.743683 10079 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting all ops after (but not including) 1622
I20260812 08:01:22.743759 10079 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 11.1623
I20260812 08:01:22.744076 10079 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 11.1624
I20260812 08:01:22.744272 10079 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 11.1625
I20260812 08:01:22.744447 10079 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 11.1626
I20260812 08:01:22.744616 10079 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 11.1627
I20260812 08:01:22.745406 10577 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1623, Last known committed idx: 1619, Time since last communication: 0.000s
I20260812 08:01:22.794924 10460 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 12 FOLLOWER]: Refusing update from remote peer 6801d45ca6614156906e5d571ebcb8a6: Log matching property violated. Preceding OpId in replica: term: 10 index: 1408. Preceding OpId from leader: term: 12 index: 1623. (index mismatch)
I20260812 08:01:22.795594 10577 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1623, Last known committed idx: 1399, Time since last communication: 0.000s
I20260812 08:01:22.829319  8004 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 12 FOLLOWER]: Refusing update from remote peer 6801d45ca6614156906e5d571ebcb8a6: Log matching property violated. Preceding OpId in replica: term: 11 index: 1564. Preceding OpId from leader: term: 12 index: 1623. (index mismatch)
I20260812 08:01:22.829994 10577 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1623, Last known committed idx: 1562, Time since last communication: 0.000s
I20260812 08:01:22.850154  9809 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 12 FOLLOWER]: Refusing update from remote peer 6801d45ca6614156906e5d571ebcb8a6: Log matching property violated. Preceding OpId in replica: term: 11 index: 1627. Preceding OpId from leader: term: 12 index: 1623. (term mismatch)
I20260812 08:01:22.850274  9809 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting all ops after (but not including) 1622
I20260812 08:01:22.850356  9809 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting uncommitted WRITE_OP operation due to leader change: 11.1623
I20260812 08:01:22.850664  9809 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48992 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1614, attempt_no=1}) took 1559 ms. Trace:
I20260812 08:01:22.850730  9809 rpcz_store.cc:276] 0812 08:01:21.290776 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:21.290809 (+    33us) service_pool.cc:224] Handling call
0812 08:01:22.850647 (+1559838us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 08:01:22.850984  9809 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting uncommitted WRITE_OP operation due to leader change: 11.1624
I20260812 08:01:22.851150  9809 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48992 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1615, attempt_no=1}) took 1559 ms. Trace:
I20260812 08:01:22.851195  9809 rpcz_store.cc:276] 0812 08:01:21.291393 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:21.291418 (+    25us) service_pool.cc:224] Handling call
0812 08:01:22.851139 (+1559721us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 08:01:22.851327  9809 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting uncommitted WRITE_OP operation due to leader change: 11.1625
I20260812 08:01:22.851454  9809 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48992 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1616, attempt_no=1}) took 1560 ms. Trace:
I20260812 08:01:22.851496  9809 rpcz_store.cc:276] 0812 08:01:21.290605 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:21.290674 (+    69us) service_pool.cc:224] Handling call
0812 08:01:22.851445 (+1560771us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 08:01:22.851622  9809 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting uncommitted WRITE_OP operation due to leader change: 11.1626
I20260812 08:01:22.851745  9809 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48992 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1617, attempt_no=1}) took 1560 ms. Trace:
I20260812 08:01:22.851785  9809 rpcz_store.cc:276] 0812 08:01:21.291258 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:21.291288 (+    30us) service_pool.cc:224] Handling call
0812 08:01:22.851738 (+1560450us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 08:01:22.851922  9809 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting uncommitted WRITE_OP operation due to leader change: 11.1627
I20260812 08:01:22.852064  9809 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48992 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1618, attempt_no=1}) took 1560 ms. Trace:
I20260812 08:01:22.852104  9809 rpcz_store.cc:276] 0812 08:01:21.291332 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:21.291375 (+    43us) service_pool.cc:224] Handling call
0812 08:01:22.852057 (+1560682us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260812 08:01:22.859321  9743 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48992: Illegal state: replica 1664e32d3cd0473592085aa522379f22 is not leader of this config: current role FOLLOWER
W20260812 08:01:22.859416  9741 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48992: Illegal state: replica 1664e32d3cd0473592085aa522379f22 is not leader of this config: current role FOLLOWER
W20260812 08:01:22.859902  9744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48992: Illegal state: replica 1664e32d3cd0473592085aa522379f22 is not leader of this config: current role FOLLOWER
W20260812 08:01:22.860343  9748 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48992: Illegal state: replica 1664e32d3cd0473592085aa522379f22 is not leader of this config: current role FOLLOWER
W20260812 08:01:22.860805  9751 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48992: Illegal state: replica 1664e32d3cd0473592085aa522379f22 is not leader of this config: current role FOLLOWER
I20260812 08:01:22.862214 10577 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1623, Last known committed idx: 1619, Time since last communication: 0.000s
I20260812 08:01:22.887629 10886 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48992 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1611, attempt_no=1}) took 1596 ms. Trace:
I20260812 08:01:22.887853 10886 rpcz_store.cc:276] 0812 08:01:21.291014 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:21.291060 (+    46us) service_pool.cc:224] Handling call
0812 08:01:22.887605 (+1596545us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:01:22.889853 10886 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48992 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1612, attempt_no=1}) took 1599 ms. Trace:
I20260812 08:01:22.890017 10886 rpcz_store.cc:276] 0812 08:01:21.290707 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:21.290749 (+    42us) service_pool.cc:224] Handling call
0812 08:01:22.889838 (+1599089us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:01:22.891256 10886 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48992 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1613, attempt_no=1}) took 1600 ms. Trace:
I20260812 08:01:22.891374 10886 rpcz_store.cc:276] 0812 08:01:21.291083 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:21.291114 (+    31us) service_pool.cc:224] Handling call
0812 08:01:22.891243 (+1600129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:01:22.909417 10854 mvcc.cc:204] Tried to move back new op lower bound from 7317592804551135232 to 7317592798302756864. Current Snapshot: MvccSnapshot[applied={T|T < 7317592802911150080 or (T in {7317592802913083392,7317592802914942976,7317592802993741824,7317592802995949568,7317592803042410496,7317592803044564992,7317592803046526976,7317592803048460288,7317592803050336256,7317592803135909888})}]
W20260812 08:01:22.910512  8116 consensus_peers.cc:597] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 -> Peer 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363): Couldn't send request to peer 56b60424ed9f4a079689ad82a318d29a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:01:23.526245 10820 heartbeater.cc:499] Master 127.5.142.190:44379 was elected leader, sending a full tablet report...
I20260812 08:01:24.482388 10635 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1634 overwritten=7 applied=1619 ignored=0} inserts{seen=16110 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 8 replicates
I20260812 08:01:24.482969 10635 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Bootstrap complete.
I20260812 08:01:24.484313 10635 ts_tablet_manager.cc:1403] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Time spent bootstrapping tablet: real 1.987s	user 1.401s	sys 0.080s
I20260812 08:01:24.486747 10635 raft_consensus.cc:359] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 11 FOLLOWER]: Replica starting. Triggering 8 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:24.488823 10635 raft_consensus.cc:740] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 11 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56b60424ed9f4a079689ad82a318d29a, State: Initialized, Role: FOLLOWER
I20260812 08:01:24.489313 10635 consensus_queue.cc:260] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1619, Last appended: 11.1627, Last appended by leader: 1627, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:24.490125 10635 ts_tablet_manager.cc:1434] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260812 08:01:25.193545 10919 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6801d45ca6614156906e5d571ebcb8a6)
I20260812 08:01:25.193723 10919 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:25.195031 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 13 candidate_status { last_received { term: 12 index: 1767 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:25.195055 10082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 13 candidate_status { last_received { term: 12 index: 1767 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:25.195343 10919 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:25.195724  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 13 candidate_status { last_received { term: 12 index: 1767 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:25.200105  9808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 13 candidate_status { last_received { term: 12 index: 1767 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:25.200596  7286 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 16901c5559a748a9aa770d73e254099c; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 780d7e64a8214418a371f35351b113df, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:25.200802 10919 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260812 08:01:25.204370 10738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 13 candidate_status { last_received { term: 12 index: 1767 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:25.204715 10738 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16901c5559a748a9aa770d73e254099c in term 11.
I20260812 08:01:25.426887 10923 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6801d45ca6614156906e5d571ebcb8a6)
I20260812 08:01:25.427052 10923 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:25.427654 10925 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6801d45ca6614156906e5d571ebcb8a6)
I20260812 08:01:25.427778 10925 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:25.428701 10923 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:25.432411 10925 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:25.435819  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 13 candidate_status { last_received { term: 12 index: 1854 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:25.435904 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 13 candidate_status { last_received { term: 12 index: 1854 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:25.435983  7381 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 12.
I20260812 08:01:25.436046 10460 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 12.
I20260812 08:01:25.438486  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 13 candidate_status { last_received { term: 12 index: 1854 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:25.438862  9808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 13 candidate_status { last_received { term: 12 index: 1854 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:25.439203 10082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 13 candidate_status { last_received { term: 12 index: 1854 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:25.439330 10082 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 12.
I20260812 08:01:25.439913 10738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 13 candidate_status { last_received { term: 12 index: 1854 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:25.440057 10738 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 11.
I20260812 08:01:25.440430 10364 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, faafbc72cbd444ff90977a2cdb091b72; no voters: 780d7e64a8214418a371f35351b113df
I20260812 08:01:25.440495  9808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 13 candidate_status { last_received { term: 12 index: 1854 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:25.440804 10923 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260812 08:01:25.440867 10923 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6801d45ca6614156906e5d571ebcb8a6)
I20260812 08:01:25.440932 10923 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 12 FOLLOWER]: Advancing to term 13
I20260812 08:01:25.441941 10923 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:25.443935  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 13 candidate_status { last_received { term: 12 index: 1854 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:25.444082  7381 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 12.
I20260812 08:01:25.444154 10082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 13 candidate_status { last_received { term: 12 index: 1854 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72"
I20260812 08:01:25.444329 10082 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 12 FOLLOWER]: Advancing to term 13
I20260812 08:01:25.445209 10738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 13 candidate_status { last_received { term: 12 index: 1854 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:25.445354 10738 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 11.
I20260812 08:01:25.445514 10082 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 13.
I20260812 08:01:25.445740  9966 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, faafbc72cbd444ff90977a2cdb091b72; no voters: 1664e32d3cd0473592085aa522379f22
I20260812 08:01:25.446012 10925 raft_consensus.cc:2764] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260812 08:01:25.446452  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 13 candidate_status { last_received { term: 12 index: 1854 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:25.446725  9808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 13 candidate_status { last_received { term: 12 index: 1854 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22"
I20260812 08:01:25.446987 10738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 13 candidate_status { last_received { term: 12 index: 1854 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a"
I20260812 08:01:25.447136 10923 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 13 election: Requested vote from peers faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:25.447189 10738 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 11 FOLLOWER]: Advancing to term 13
I20260812 08:01:25.447556  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 13 candidate_status { last_received { term: 12 index: 1854 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:01:25.447676  7381 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 12 FOLLOWER]: Advancing to term 13
I20260812 08:01:25.447978  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 13 candidate_status { last_received { term: 12 index: 1854 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:01:25.448761  7381 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 13.
I20260812 08:01:25.449848 10738 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 13.
I20260812 08:01:25.450119 10364 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, faafbc72cbd444ff90977a2cdb091b72; no voters: 1664e32d3cd0473592085aa522379f22, 780d7e64a8214418a371f35351b113df
I20260812 08:01:25.450306 10923 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 13 FOLLOWER]: Leader election won for term 13
I20260812 08:01:25.450526 10923 raft_consensus.cc:697] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 13 LEADER]: Becoming Leader. State: Replica: 3ae3f7a6e9f943e996a6dcea504878d0, State: Running, Role: LEADER
I20260812 08:01:25.450846 10923 consensus_queue.cc:237] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1846, Committed index: 1846, Last appended: 12.1854, Last appended by leader: 1853, Current term: 13, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:25.453866  6748 catalog_manager.cc:5719] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 reported cstate change: term changed from 12 to 13, leader changed from 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135) to 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129). New cstate: current_term: 13 leader_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } health_report { overall_health: UNKNOWN } } }
W20260812 08:01:25.509372  8112 connection.cc:256] RPC call timeout handler was delayed by -1.0222253892570734s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 08:01:25.511494 10891 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:39074 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1837, attempt_no=0}) took 1604 ms. Trace:
I20260812 08:01:25.511509 10082 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 13 FOLLOWER]: Rejecting Update request from peer 6801d45ca6614156906e5d571ebcb8a6 for earlier term 12. Current term is 13. Ops: []
I20260812 08:01:25.511575 10891 rpcz_store.cc:276] 0812 08:01:23.907338 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:23.907402 (+    64us) service_pool.cc:224] Handling call
0812 08:01:25.511470 (+1604068us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:23.916040 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:23.916115 (+    75us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:23.916118 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:23.916119 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:23.916207 (+    88us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:23.916212 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:23.916215 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:23.916276 (+    61us) tablet.cc:701] Row locks acquired
0812 08:01:23.916277 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:23.916304 (+    27us) write_op.cc:270] Start()
0812 08:01:23.916322 (+    18us) write_op.cc:276] Timestamp: P: 1786521683916282 usec, L: 0
0812 08:01:23.916324 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:23.916425 (+   101us) log.cc:844] Serialized 696 byte log entry
0812 08:01:25.510543 (+1594118us) op_driver.cc:464] REPLICATION: finished
0812 08:01:25.510837 (+   294us) write_op.cc:301] APPLY: starting
0812 08:01:25.510857 (+    20us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:25.510899 (+    42us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:25.510911 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:25.511099 (+   188us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:25.511133 (+    34us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:25.511138 (+     5us) write_op.cc:312] APPLY: finished
0812 08:01:25.511220 (+    82us) log.cc:844] Serialized 77 byte log entry
0812 08:01:25.511377 (+   157us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:25.511395 (+    18us) write_op.cc:454] Released schema lock
0812 08:01:25.511402 (+     7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":235,"mutex_wait_us":1,"num_ops":10,"prepare.queue_time_us":8400,"prepare.run_cpu_time_us":457,"prepare.run_wall_time_us":500,"raft.queue_time_us":29,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":5,"replication_time_us":1594186,"thread_start_us":91,"threads_started":1}]]}
I20260812 08:01:25.512382 10867 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 }, Status: INVALID_TERM, Last received: 12.1854, Next index: 1855, Last known committed idx: 1846, Time since last communication: 0.000s
I20260812 08:01:25.512971 10891 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:39074 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1838, attempt_no=0}) took 1605 ms. Trace:
I20260812 08:01:25.513031 10891 rpcz_store.cc:276] 0812 08:01:23.907424 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:23.907471 (+    47us) service_pool.cc:224] Handling call
0812 08:01:25.512959 (+1605488us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:23.916554 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:23.916624 (+    70us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:23.916628 (+     4us) write_op.cc:435] Acquired schema lock
0812 08:01:23.916629 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:23.916709 (+    80us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:23.916714 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:23.916717 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:23.916773 (+    56us) tablet.cc:701] Row locks acquired
0812 08:01:23.916774 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:23.916803 (+    29us) write_op.cc:270] Start()
0812 08:01:23.916822 (+    19us) write_op.cc:276] Timestamp: P: 1786521683916779 usec, L: 0
0812 08:01:23.916824 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:23.918289 (+  1465us) log.cc:844] Serialized 696 byte log entry
0812 08:01:25.510593 (+1592304us) op_driver.cc:464] REPLICATION: finished
0812 08:01:25.512519 (+  1926us) write_op.cc:301] APPLY: starting
0812 08:01:25.512530 (+    11us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:25.512563 (+    33us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:25.512574 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:25.512738 (+   164us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:25.512765 (+    27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:25.512769 (+     4us) write_op.cc:312] APPLY: finished
0812 08:01:25.512827 (+    58us) log.cc:844] Serialized 77 byte log entry
0812 08:01:25.512861 (+    34us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:25.512872 (+    11us) write_op.cc:454] Released schema lock
0812 08:01:25.512877 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1898,"num_ops":10,"prepare.queue_time_us":8784,"prepare.run_cpu_time_us":474,"prepare.run_wall_time_us":1851,"raft.queue_time_us":28,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":5,"replication_time_us":1593759,"threads_started":1}]]}
I20260812 08:01:25.513942  7381 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 13 FOLLOWER]: Rejecting Update request from peer 6801d45ca6614156906e5d571ebcb8a6 for earlier term 12. Current term is 13. Ops: [12.1768-12.1854]
I20260812 08:01:25.515008 10460 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 13 LEADER]: Rejecting Update request from peer 6801d45ca6614156906e5d571ebcb8a6 for earlier term 12. Current term is 13. Ops: []
I20260812 08:01:25.515614 10870 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 }, Status: INVALID_TERM, Last received: 12.1854, Next index: 1855, Last known committed idx: 1846, Time since last communication: 0.000s
I20260812 08:01:25.515805 10870 consensus_queue.cc:1243] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [LEADER]: Peer 3ae3f7a6e9f943e996a6dcea504878d0 log is divergent from this leader: its last log entry 13.1855 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1846
I20260812 08:01:25.515863 10867 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 }, Status: INVALID_TERM, Last received: 12.1767, Next index: 1768, Last known committed idx: 1765, Time since last communication: 0.000s
I20260812 08:01:25.517434  8224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 13 candidate_status { last_received { term: 12 index: 1767 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:25.519110 10900 raft_consensus.cc:3055] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 12 LEADER]: Stepping down as leader of term 12
I20260812 08:01:25.519183 10900 raft_consensus.cc:740] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6801d45ca6614156906e5d571ebcb8a6, State: Running, Role: LEADER
I20260812 08:01:25.519342 10900 consensus_queue.cc:260] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1850, Committed index: 1850, Last appended: 12.1854, Last appended by leader: 1854, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
W20260812 08:01:25.519532  8145 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39074: Illegal state: replica 6801d45ca6614156906e5d571ebcb8a6 is not leader of this config: current role FOLLOWER
I20260812 08:01:25.520293 10891 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:39074 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1839, attempt_no=0}) took 1606 ms. Trace:
I20260812 08:01:25.520555 10082 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 13 FOLLOWER]: Rejecting Update request from peer 6801d45ca6614156906e5d571ebcb8a6 for earlier term 12. Current term is 13. Ops: []
I20260812 08:01:25.520360 10891 rpcz_store.cc:276] 0812 08:01:23.913562 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:23.913611 (+    49us) service_pool.cc:224] Handling call
0812 08:01:25.520284 (+1606673us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:23.918425 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:23.919035 (+   610us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:23.919038 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:23.919039 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:23.919132 (+    93us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:23.919138 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:23.919141 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:23.919201 (+    60us) tablet.cc:701] Row locks acquired
0812 08:01:23.919203 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:23.919232 (+    29us) write_op.cc:270] Start()
0812 08:01:23.919250 (+    18us) write_op.cc:276] Timestamp: P: 1786521683919208 usec, L: 0
0812 08:01:23.919252 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:23.919355 (+   103us) log.cc:844] Serialized 696 byte log entry
0812 08:01:25.517832 (+1598477us) op_driver.cc:464] REPLICATION: finished
0812 08:01:25.519860 (+  2028us) write_op.cc:301] APPLY: starting
0812 08:01:25.519873 (+    13us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:25.519909 (+    36us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:25.519920 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:25.520099 (+   179us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:25.520126 (+    27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:25.520130 (+     4us) write_op.cc:312] APPLY: finished
0812 08:01:25.520186 (+    56us) log.cc:844] Serialized 77 byte log entry
0812 08:01:25.520217 (+    31us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:25.520233 (+    16us) write_op.cc:454] Released schema lock
0812 08:01:25.520238 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1991,"num_ops":10,"prepare.queue_time_us":4563,"prepare.run_cpu_time_us":472,"prepare.run_wall_time_us":1002,"raft.queue_time_us":34,"raft.run_cpu_time_us":7,"raft.run_wall_time_us":9,"replication_time_us":1598563}]]}
I20260812 08:01:25.521142 10867 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 }, Status: INVALID_TERM, Last received: 12.1854, Next index: 1855, Last known committed idx: 1846, Time since last communication: 0.000s
I20260812 08:01:25.521919  7381 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 13 FOLLOWER]: Rejecting Update request from peer 6801d45ca6614156906e5d571ebcb8a6 for earlier term 12. Current term is 13. Ops: [12.1768-12.1854]
I20260812 08:01:25.522163 10900 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 12 FOLLOWER]: Advancing to term 13
I20260812 08:01:25.524178 10460 raft_consensus.cc:1217] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 13 LEADER]: Deduplicated request from leader. Original: 12.1846->[12.1847-12.1854]   Dedup: 12.1854->[]
I20260812 08:01:25.524343 10460 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 13 LEADER]: Rejecting Update request from peer 6801d45ca6614156906e5d571ebcb8a6 for earlier term 12. Current term is 13. Ops: [12.1847-12.1854]
I20260812 08:01:25.525192 10891 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:39074 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1840, attempt_no=0}) took 1611 ms. Trace:
I20260812 08:01:25.525288 10891 rpcz_store.cc:276] 0812 08:01:23.913643 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:23.913679 (+    36us) service_pool.cc:224] Handling call
0812 08:01:25.525178 (+1611499us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:23.919451 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:23.919521 (+    70us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:23.919524 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:23.919525 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:23.919606 (+    81us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:23.919612 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:23.919615 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:23.919673 (+    58us) tablet.cc:701] Row locks acquired
0812 08:01:23.919674 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:23.919702 (+    28us) write_op.cc:270] Start()
0812 08:01:23.919720 (+    18us) write_op.cc:276] Timestamp: P: 1786521683919678 usec, L: 0
0812 08:01:23.919722 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:23.919820 (+    98us) log.cc:844] Serialized 696 byte log entry
0812 08:01:25.517879 (+1598059us) op_driver.cc:464] REPLICATION: finished
0812 08:01:25.520767 (+  2888us) write_op.cc:301] APPLY: starting
0812 08:01:25.520776 (+     9us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:25.520802 (+    26us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:25.520812 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:25.524445 (+  3633us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:25.524498 (+    53us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:25.524504 (+     6us) write_op.cc:312] APPLY: finished
0812 08:01:25.524603 (+    99us) log.cc:844] Serialized 77 byte log entry
0812 08:01:25.525038 (+   435us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:25.525054 (+    16us) write_op.cc:454] Released schema lock
0812 08:01:25.525060 (+     6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2866,"num_ops":10,"prepare.queue_time_us":5565,"prepare.run_cpu_time_us":428,"prepare.run_wall_time_us":470,"raft.queue_time_us":32,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":4,"replication_time_us":1598147}]]}
W20260812 08:01:25.527770  9751 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48992: Illegal state: replica 1664e32d3cd0473592085aa522379f22 is not leader of this config: current role FOLLOWER
W20260812 08:01:25.528770  8145 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39074: Illegal state: replica 6801d45ca6614156906e5d571ebcb8a6 is not leader of this config: current role FOLLOWER
I20260812 08:01:25.529137  8220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 13 candidate_status { last_received { term: 12 index: 1854 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:25.529275  8220 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 13.
W20260812 08:01:25.530692  9751 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48992: Illegal state: replica 1664e32d3cd0473592085aa522379f22 is not leader of this config: current role FOLLOWER
W20260812 08:01:25.531082 10983 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39074: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.012s ago.
W20260812 08:01:25.534516  9751 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48992: Illegal state: replica 1664e32d3cd0473592085aa522379f22 is not leader of this config: current role FOLLOWER
W20260812 08:01:25.534641  7321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:40918: Illegal state: replica 16901c5559a748a9aa770d73e254099c is not leader of this config: current role FOLLOWER
I20260812 08:01:25.535360  8220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 13 candidate_status { last_received { term: 12 index: 1854 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:25.535426  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 13 candidate_status { last_received { term: 12 index: 1854 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
I20260812 08:01:25.535497  8220 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 13.
W20260812 08:01:25.536250  7321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:40918: Illegal state: replica 16901c5559a748a9aa770d73e254099c is not leader of this config: current role FOLLOWER
W20260812 08:01:25.537775  7321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:40918: Illegal state: replica 16901c5559a748a9aa770d73e254099c is not leader of this config: current role FOLLOWER
W20260812 08:01:25.542295  7321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:40918: Illegal state: replica 16901c5559a748a9aa770d73e254099c is not leader of this config: current role FOLLOWER
W20260812 08:01:25.542301  7935 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35090: Illegal state: replica 780d7e64a8214418a371f35351b113df is not leader of this config: current role FOLLOWER
W20260812 08:01:25.543434  7935 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35090: Illegal state: replica 780d7e64a8214418a371f35351b113df is not leader of this config: current role FOLLOWER
W20260812 08:01:25.543435  7939 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35090: Illegal state: replica 780d7e64a8214418a371f35351b113df is not leader of this config: current role FOLLOWER
W20260812 08:01:25.548237  7939 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35090: Illegal state: replica 780d7e64a8214418a371f35351b113df is not leader of this config: current role FOLLOWER
W20260812 08:01:25.548244  9751 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48992: Illegal state: replica 1664e32d3cd0473592085aa522379f22 is not leader of this config: current role FOLLOWER
W20260812 08:01:25.550931  8145 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39074: Illegal state: replica 6801d45ca6614156906e5d571ebcb8a6 is not leader of this config: current role FOLLOWER
I20260812 08:01:25.564144  7381 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 13 FOLLOWER]: Refusing update from remote peer 3ae3f7a6e9f943e996a6dcea504878d0: Log matching property violated. Preceding OpId in replica: term: 12 index: 1767. Preceding OpId from leader: term: 13 index: 1856. (index mismatch)
I20260812 08:01:25.564801 10923 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1855, Last known committed idx: 1765, Time since last communication: 0.000s
I20260812 08:01:25.574486  8004 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 12 FOLLOWER]: Advancing to term 13
I20260812 08:01:25.575088  8220 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 13 FOLLOWER]: Refusing update from remote peer 3ae3f7a6e9f943e996a6dcea504878d0: Log matching property violated. Preceding OpId in replica: term: 12 index: 1854. Preceding OpId from leader: term: 13 index: 1856. (index mismatch)
I20260812 08:01:25.575728 10082 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 13 FOLLOWER]: Refusing update from remote peer 3ae3f7a6e9f943e996a6dcea504878d0: Log matching property violated. Preceding OpId in replica: term: 12 index: 1854. Preceding OpId from leader: term: 13 index: 1856. (index mismatch)
I20260812 08:01:25.575776  8004 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 13 FOLLOWER]: Refusing update from remote peer 3ae3f7a6e9f943e996a6dcea504878d0: Log matching property violated. Preceding OpId in replica: term: 12 index: 1854. Preceding OpId from leader: term: 13 index: 1856. (index mismatch)
I20260812 08:01:25.576205 10738 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 13 FOLLOWER]: Refusing update from remote peer 3ae3f7a6e9f943e996a6dcea504878d0: Log matching property violated. Preceding OpId in replica: term: 11 index: 1627. Preceding OpId from leader: term: 13 index: 1856. (index mismatch)
I20260812 08:01:25.576267 10923 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1855, Last known committed idx: 1850, Time since last communication: 0.000s
I20260812 08:01:25.576458 10967 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1855, Last known committed idx: 1846, Time since last communication: 0.000s
I20260812 08:01:25.577147 10967 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1855, Last known committed idx: 1619, Time since last communication: 0.000s
I20260812 08:01:25.577271 10967 consensus_queue.cc:1243] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Peer 56b60424ed9f4a079689ad82a318d29a log is divergent from this leader: its last log entry 11.1627 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1619
I20260812 08:01:25.577666  9808 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 12 FOLLOWER]: Advancing to term 13
I20260812 08:01:25.578917  9808 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 13 FOLLOWER]: Refusing update from remote peer 3ae3f7a6e9f943e996a6dcea504878d0: Log matching property violated. Preceding OpId in replica: term: 12 index: 1854. Preceding OpId from leader: term: 13 index: 1856. (index mismatch)
I20260812 08:01:25.580142 10967 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1855, Last known committed idx: 1850, Time since last communication: 0.000s
I20260812 08:01:25.583526 10923 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1855, Last known committed idx: 1850, Time since last communication: 0.000s
I20260812 08:01:25.588253 10985 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:39074 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1841, attempt_no=0}) took 1670 ms. Trace:
I20260812 08:01:25.588351 10985 rpcz_store.cc:276] 0812 08:01:23.918027 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:23.918090 (+    63us) service_pool.cc:224] Handling call
0812 08:01:25.588232 (+1670142us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:23.920527 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:23.920596 (+    69us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:23.920599 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:23.920601 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:23.920683 (+    82us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:23.920690 (+     7us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:23.920692 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:23.921905 (+  1213us) tablet.cc:701] Row locks acquired
0812 08:01:23.921907 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:23.921934 (+    27us) write_op.cc:270] Start()
0812 08:01:23.921952 (+    18us) write_op.cc:276] Timestamp: P: 1786521683921912 usec, L: 0
0812 08:01:23.921953 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:01:23.923306 (+  1353us) log.cc:844] Serialized 696 byte log entry
0812 08:01:25.586710 (+1663404us) op_driver.cc:464] REPLICATION: finished
0812 08:01:25.587702 (+   992us) write_op.cc:301] APPLY: starting
0812 08:01:25.587715 (+    13us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:25.587752 (+    37us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:25.587766 (+    14us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:25.588005 (+   239us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:25.588039 (+    34us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:25.588045 (+     6us) write_op.cc:312] APPLY: finished
0812 08:01:25.588108 (+    63us) log.cc:844] Serialized 77 byte log entry
0812 08:01:25.588148 (+    40us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:25.588162 (+    14us) write_op.cc:454] Released schema lock
0812 08:01:25.588168 (+     6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":939,"num_ops":10,"prepare.queue_time_us":2177,"prepare.run_cpu_time_us":500,"prepare.run_wall_time_us":2896,"raft.queue_time_us":27,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":4,"replication_time_us":1664730}]]}
I20260812 08:01:25.589260 10985 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:39074 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1842, attempt_no=0}) took 1670 ms. Trace:
I20260812 08:01:25.589339 10985 rpcz_store.cc:276] 0812 08:01:23.918524 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:23.918974 (+   450us) service_pool.cc:224] Handling call
0812 08:01:25.589245 (+1670271us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:23.923436 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:23.923508 (+    72us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:23.923511 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:23.923512 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:23.923593 (+    81us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:23.923599 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:23.923602 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:23.923941 (+   339us) tablet.cc:701] Row locks acquired
0812 08:01:23.923942 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:23.923968 (+    26us) write_op.cc:270] Start()
0812 08:01:23.923986 (+    18us) write_op.cc:276] Timestamp: P: 1786521683923946 usec, L: 0
0812 08:01:23.923988 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:23.924099 (+   111us) log.cc:844] Serialized 696 byte log entry
0812 08:01:25.586779 (+1662680us) op_driver.cc:464] REPLICATION: finished
0812 08:01:25.588768 (+  1989us) write_op.cc:301] APPLY: starting
0812 08:01:25.588779 (+    11us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:25.588808 (+    29us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:25.588818 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:25.589044 (+   226us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:25.589072 (+    28us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:25.589077 (+     5us) write_op.cc:312] APPLY: finished
0812 08:01:25.589139 (+    62us) log.cc:844] Serialized 77 byte log entry
0812 08:01:25.589175 (+    36us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:25.589187 (+    12us) write_op.cc:454] Released schema lock
0812 08:01:25.589192 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1957,"num_ops":10,"prepare.queue_time_us":4217,"prepare.run_cpu_time_us":461,"prepare.run_wall_time_us":762,"raft.queue_time_us":30,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":4,"replication_time_us":1662776}]]}
I20260812 08:01:25.590137 10985 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:39074 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1843, attempt_no=0}) took 1666 ms. Trace:
I20260812 08:01:25.590193 10985 rpcz_store.cc:276] 0812 08:01:23.923243 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:23.923309 (+    66us) service_pool.cc:224] Handling call
0812 08:01:25.590126 (+1666817us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:23.924215 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:23.924285 (+    70us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:23.924288 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:23.924289 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:23.924366 (+    77us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:23.924371 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:23.924374 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:23.924432 (+    58us) tablet.cc:701] Row locks acquired
0812 08:01:23.924433 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:23.924458 (+    25us) write_op.cc:270] Start()
0812 08:01:23.924476 (+    18us) write_op.cc:276] Timestamp: P: 1786521683924435 usec, L: 0
0812 08:01:23.924478 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:23.924579 (+   101us) log.cc:844] Serialized 686 byte log entry
0812 08:01:25.586824 (+1662245us) op_driver.cc:464] REPLICATION: finished
0812 08:01:25.589689 (+  2865us) write_op.cc:301] APPLY: starting
0812 08:01:25.589700 (+    11us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:25.589727 (+    27us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:25.589739 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:25.589937 (+   198us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:25.589973 (+    36us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:25.589978 (+     5us) write_op.cc:312] APPLY: finished
0812 08:01:25.590026 (+    48us) log.cc:844] Serialized 77 byte log entry
0812 08:01:25.590059 (+    33us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:25.590071 (+    12us) write_op.cc:454] Released schema lock
0812 08:01:25.590076 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2835,"num_ops":10,"prepare.queue_time_us":659,"prepare.run_cpu_time_us":420,"prepare.run_wall_time_us":463,"raft.queue_time_us":29,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":5,"replication_time_us":1662332,"thread_start_us":153,"threads_started":1}]]}
I20260812 08:01:25.590987 10985 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:39074 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=1844, attempt_no=0}) took 1667 ms. Trace:
I20260812 08:01:25.591055 10985 rpcz_store.cc:276] 0812 08:01:23.923850 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:23.924124 (+   274us) service_pool.cc:224] Handling call
0812 08:01:25.590976 (+1666852us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:23.924695 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:23.924764 (+    69us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:23.924767 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:23.924769 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:23.924874 (+   105us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:23.924879 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:23.924882 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:23.925229 (+   347us) tablet.cc:701] Row locks acquired
0812 08:01:23.925231 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:23.925261 (+    30us) write_op.cc:270] Start()
0812 08:01:23.925281 (+    20us) write_op.cc:276] Timestamp: P: 1786521683925238 usec, L: 0
0812 08:01:23.925283 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:23.925384 (+   101us) log.cc:844] Serialized 686 byte log entry
0812 08:01:25.587052 (+1661668us) op_driver.cc:464] REPLICATION: finished
0812 08:01:25.590538 (+  3486us) write_op.cc:301] APPLY: starting
0812 08:01:25.590548 (+    10us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:25.590576 (+    28us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:25.590587 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:25.590773 (+   186us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:25.590798 (+    25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:25.590804 (+     6us) write_op.cc:312] APPLY: finished
0812 08:01:25.590857 (+    53us) log.cc:844] Serialized 77 byte log entry
0812 08:01:25.590895 (+    38us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:25.590907 (+    12us) write_op.cc:454] Released schema lock
0812 08:01:25.590912 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3453,"num_ops":10,"prepare.queue_time_us":345,"prepare.run_cpu_time_us":491,"prepare.run_wall_time_us":1008,"raft.queue_time_us":21,"raft.run_cpu_time_us":3,"raft.run_wall_time_us":4,"replication_time_us":1661746,"thread_start_us":91,"threads_started":1}]]}
I20260812 08:01:25.601238 11010 mvcc.cc:204] Tried to move back new op lower bound from 7317592824064376832 to 7317592823610916864. Current Snapshot: MvccSnapshot[applied={T|T < 7317592817323126784}]
I20260812 08:01:25.601677 10738 raft_consensus.cc:1217] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 13 FOLLOWER]: Deduplicated request from leader. Original: 11.1619->[11.1620-13.1856]   Dedup: 11.1622->[12.1623-13.1856]
I20260812 08:01:25.602150 10738 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting all ops after (but not including) 1622
I20260812 08:01:25.602288 10738 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting uncommitted WRITE_OP operation due to leader change: 11.1623
I20260812 08:01:25.602876 10738 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting uncommitted WRITE_OP operation due to leader change: 11.1624
I20260812 08:01:25.603138 10738 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting uncommitted WRITE_OP operation due to leader change: 11.1625
I20260812 08:01:25.603343 10738 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting uncommitted WRITE_OP operation due to leader change: 11.1626
I20260812 08:01:25.603543 10738 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting uncommitted WRITE_OP operation due to leader change: 11.1627
I20260812 08:01:25.836038 11046 mvcc.cc:204] Tried to move back new op lower bound from 7317592824064376832 to 7317592810766278656. Current Snapshot: MvccSnapshot[applied={T|T < 7317592824064376832}]
I20260812 08:01:27.649302 11083 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3ae3f7a6e9f943e996a6dcea504878d0)
I20260812 08:01:27.649458 11083 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:27.650317 11083 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183)
I20260812 08:01:27.650744  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 14 candidate_status { last_received { term: 13 index: 1967 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:27.650743  9809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 14 candidate_status { last_received { term: 13 index: 1967 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:27.650748 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 14 candidate_status { last_received { term: 13 index: 1967 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:27.651397  7375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 14 candidate_status { last_received { term: 13 index: 1967 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:27.651819  7907 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:27.651978 11083 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260812 08:01:27.654707 10738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 14 candidate_status { last_received { term: 13 index: 1967 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:27.781535 11088 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3ae3f7a6e9f943e996a6dcea504878d0)
I20260812 08:01:27.781698 11088 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:27.782563 11088 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:27.783046  9809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 14 candidate_status { last_received { term: 13 index: 2006 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:27.783034  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 14 candidate_status { last_received { term: 13 index: 2006 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:27.783077 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 14 candidate_status { last_received { term: 13 index: 2006 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:27.783048  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 14 candidate_status { last_received { term: 13 index: 2006 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:27.783209  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16901c5559a748a9aa770d73e254099c in term 13.
I20260812 08:01:27.783718 10738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 14 candidate_status { last_received { term: 13 index: 2006 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:27.784117  7286 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:27.784307 11088 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260812 08:01:27.901034 11089 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3ae3f7a6e9f943e996a6dcea504878d0)
I20260812 08:01:27.901211 11089 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:27.901995 11089 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:27.902318 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 2044 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:27.902357  7375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 2044 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:27.902479  7375 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6801d45ca6614156906e5d571ebcb8a6 in term 13.
I20260812 08:01:27.902470 10738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 2044 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:27.902715  9809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 2044 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:27.903002  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 2044 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:27.903147  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6801d45ca6614156906e5d571ebcb8a6 in term 13.
I20260812 08:01:28.010278 11090 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3ae3f7a6e9f943e996a6dcea504878d0)
I20260812 08:01:28.010442 11090 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:28.011860 11090 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:28.016687  9809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 14 candidate_status { last_received { term: 13 index: 2093 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:28.016923  9809 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56b60424ed9f4a079689ad82a318d29a for term 14 because replica has last-logged OpId of term: 13 index: 2096, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 2093.
I20260812 08:01:28.017242  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 14 candidate_status { last_received { term: 13 index: 2093 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:28.017400  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 13.
I20260812 08:01:28.017648 11111 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3ae3f7a6e9f943e996a6dcea504878d0)
I20260812 08:01:28.017742 11111 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:28.017894 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 14 candidate_status { last_received { term: 13 index: 2093 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:28.018054 10079 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56b60424ed9f4a079689ad82a318d29a for term 14 because replica has last-logged OpId of term: 13 index: 2096, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 2093.
I20260812 08:01:28.018627 11112 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3ae3f7a6e9f943e996a6dcea504878d0)
I20260812 08:01:28.018851 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 14 candidate_status { last_received { term: 13 index: 2096 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:28.018782 11112 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:28.019274  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 14 candidate_status { last_received { term: 13 index: 2096 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:28.019423  8225 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 13.
I20260812 08:01:28.019495  7375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 14 candidate_status { last_received { term: 13 index: 2096 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:28.019534 11111 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:28.019630  7375 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 13.
I20260812 08:01:28.019708 10079 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 13.
I20260812 08:01:28.020190  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 14 candidate_status { last_received { term: 13 index: 2096 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:28.020315  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 13.
I20260812 08:01:28.020366  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 14 candidate_status { last_received { term: 13 index: 2093 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:28.020486  8225 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 13.
I20260812 08:01:28.020622  9706 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 
I20260812 08:01:28.021098  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 14 candidate_status { last_received { term: 13 index: 2096 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:28.021095  7375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 14 candidate_status { last_received { term: 13 index: 2093 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:28.021199  8225 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 13.
I20260812 08:01:28.021221  7375 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 13.
I20260812 08:01:28.021328  9809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 14 candidate_status { last_received { term: 13 index: 2096 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:28.021438  9809 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 13.
I20260812 08:01:28.021559 10738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 14 candidate_status { last_received { term: 13 index: 2096 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:28.021710 10738 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 13.
I20260812 08:01:28.021850  7375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 14 candidate_status { last_received { term: 13 index: 2096 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:28.021977  7375 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 13.
I20260812 08:01:28.021927 11111 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260812 08:01:28.022050 10631 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:28.022110 11111 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3ae3f7a6e9f943e996a6dcea504878d0)
I20260812 08:01:28.022281 11111 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 13 FOLLOWER]: Advancing to term 14
I20260812 08:01:28.022289  9969 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72; no voters: 
I20260812 08:01:28.022444 11090 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260812 08:01:28.022506 11090 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3ae3f7a6e9f943e996a6dcea504878d0)
I20260812 08:01:28.022557 11090 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 13 FOLLOWER]: Advancing to term 14
I20260812 08:01:28.022876  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 14 candidate_status { last_received { term: 13 index: 2096 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:28.022997  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 13.
I20260812 08:01:28.023339 11112 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:28.023447 11112 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260812 08:01:28.023500 11112 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3ae3f7a6e9f943e996a6dcea504878d0)
I20260812 08:01:28.023540 11112 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 13 FOLLOWER]: Advancing to term 14
I20260812 08:01:28.023828 11090 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:28.023934 11111 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:28.024464 11090 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 14 election: Requested vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:28.024569 11112 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:28.024757  7375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 14 candidate_status { last_received { term: 13 index: 2093 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:01:28.024854  7375 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 13 FOLLOWER]: Advancing to term 14
I20260812 08:01:28.024919  9809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 14 candidate_status { last_received { term: 13 index: 2093 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22"
I20260812 08:01:28.024999  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 14 candidate_status { last_received { term: 13 index: 2093 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
I20260812 08:01:28.025060  9809 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 56b60424ed9f4a079689ad82a318d29a in current term 14: Already voted for candidate 1664e32d3cd0473592085aa522379f22 in this term.
I20260812 08:01:28.025017 10734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 14 candidate_status { last_received { term: 13 index: 2096 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a"
I20260812 08:01:28.025094  8225 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 13 FOLLOWER]: Advancing to term 14
I20260812 08:01:28.025179 10734 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1664e32d3cd0473592085aa522379f22 in current term 14: Already voted for candidate 56b60424ed9f4a079689ad82a318d29a in this term.
I20260812 08:01:28.025506 11111 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 14 election: Requested vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:28.025900  8220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 14 candidate_status { last_received { term: 13 index: 2096 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
I20260812 08:01:28.026151  8225 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 14.
I20260812 08:01:28.026204 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 14 candidate_status { last_received { term: 13 index: 2093 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72"
I20260812 08:01:28.026274  7375 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 14.
I20260812 08:01:28.026412 10738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 14 candidate_status { last_received { term: 13 index: 2096 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:28.026476  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 14 candidate_status { last_received { term: 13 index: 2096 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:01:28.026541 10738 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1664e32d3cd0473592085aa522379f22 in current term 14: Already voted for candidate 56b60424ed9f4a079689ad82a318d29a in this term.
I20260812 08:01:28.026598  7381 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1664e32d3cd0473592085aa522379f22 in current term 14: Already voted for candidate 56b60424ed9f4a079689ad82a318d29a in this term.
I20260812 08:01:28.026762  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 14 candidate_status { last_received { term: 13 index: 2093 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:01:28.026875  8004 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 13 FOLLOWER]: Advancing to term 14
I20260812 08:01:28.027017  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 14 candidate_status { last_received { term: 13 index: 2096 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:01:28.027012 10082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 14 candidate_status { last_received { term: 13 index: 2096 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72"
I20260812 08:01:28.027520  9706 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 1664e32d3cd0473592085aa522379f22; no voters: 16901c5559a748a9aa770d73e254099c, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df
I20260812 08:01:28.027593 10079 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 56b60424ed9f4a079689ad82a318d29a in current term 14: Already voted for candidate faafbc72cbd444ff90977a2cdb091b72 in this term.
I20260812 08:01:28.027832 11111 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260812 08:01:28.027916  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 14.
I20260812 08:01:28.028337 10623 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:28.028326  9809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 14 candidate_status { last_received { term: 13 index: 2096 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22"
I20260812 08:01:28.028455  9809 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate faafbc72cbd444ff90977a2cdb091b72 in current term 14: Already voted for candidate 1664e32d3cd0473592085aa522379f22 in this term.
I20260812 08:01:28.028515  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 14 candidate_status { last_received { term: 13 index: 2096 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
I20260812 08:01:28.028649  8225 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate faafbc72cbd444ff90977a2cdb091b72 in current term 14: Already voted for candidate 56b60424ed9f4a079689ad82a318d29a in this term.
I20260812 08:01:28.028636 10738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 14 candidate_status { last_received { term: 13 index: 2096 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a"
I20260812 08:01:28.028683  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 14 candidate_status { last_received { term: 13 index: 2096 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:01:28.028759 10738 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate faafbc72cbd444ff90977a2cdb091b72 in current term 14: Already voted for candidate 56b60424ed9f4a079689ad82a318d29a in this term.
I20260812 08:01:28.028787  7381 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate faafbc72cbd444ff90977a2cdb091b72 in current term 14: Already voted for candidate 56b60424ed9f4a079689ad82a318d29a in this term.
I20260812 08:01:28.028952 11090 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 14 FOLLOWER]: Leader election won for term 14
I20260812 08:01:28.029167 11112 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 14 election: Requested vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:28.029183 11090 raft_consensus.cc:697] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 14 LEADER]: Becoming Leader. State: Replica: 56b60424ed9f4a079689ad82a318d29a, State: Running, Role: LEADER
I20260812 08:01:28.029480  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 14 candidate_status { last_received { term: 13 index: 2096 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:01:28.029551  9966 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: faafbc72cbd444ff90977a2cdb091b72; no voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6
I20260812 08:01:28.029619  8004 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate faafbc72cbd444ff90977a2cdb091b72 in current term 14: Already voted for candidate 56b60424ed9f4a079689ad82a318d29a in this term.
I20260812 08:01:28.029556 11090 consensus_queue.cc:237] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2088, Committed index: 2088, Last appended: 13.2093, Last appended by leader: 2093, Current term: 14, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:28.029731 11112 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260812 08:01:28.032428  6748 catalog_manager.cc:5719] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a reported cstate change: term changed from 13 to 14, leader changed from 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129) to 56b60424ed9f4a079689ad82a318d29a (127.5.142.130). New cstate: current_term: 14 leader_uuid: "56b60424ed9f4a079689ad82a318d29a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } health_report { overall_health: UNKNOWN } } }
W20260812 08:01:28.102846 10363 connection.cc:256] RPC call timeout handler was delayed by -1.0624931310303509s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 08:01:28.103300 10464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 14 candidate_status { last_received { term: 13 index: 2096 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:28.103629 10466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 14 candidate_status { last_received { term: 13 index: 2096 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:28.104020 10469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 14 candidate_status { last_received { term: 13 index: 2096 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0"
I20260812 08:01:28.104380 10471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 2044 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:28.105043 10462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 14 candidate_status { last_received { term: 13 index: 2096 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0"
I20260812 08:01:28.105088 10476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 14 candidate_status { last_received { term: 13 index: 1967 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:28.105268 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 14 candidate_status { last_received { term: 13 index: 2006 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:28.106194 10079 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 14 FOLLOWER]: Rejecting Update request from peer 3ae3f7a6e9f943e996a6dcea504878d0 for earlier term 13. Current term is 14. Ops: []
I20260812 08:01:28.106649 11016 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2078, attempt_no=0}) took 1607 ms. Trace:
I20260812 08:01:28.106680 11015 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 }, Status: INVALID_TERM, Last received: 13.2096, Next index: 2097, Last known committed idx: 2088, Time since last communication: 0.000s
I20260812 08:01:28.106891 11015 raft_consensus.cc:3055] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 13 LEADER]: Stepping down as leader of term 13
I20260812 08:01:28.106945 11015 raft_consensus.cc:740] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3ae3f7a6e9f943e996a6dcea504878d0, State: Running, Role: LEADER
I20260812 08:01:28.107113 11015 consensus_queue.cc:260] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2093, Committed index: 2093, Last appended: 13.2096, Last appended by leader: 2096, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:28.107426 11015 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 13 FOLLOWER]: Advancing to term 14
I20260812 08:01:28.107618 11018 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2079, attempt_no=0}) took 1608 ms. Trace:
I20260812 08:01:28.106710 11016 rpcz_store.cc:276] 0812 08:01:26.499159 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:26.499295 (+   136us) service_pool.cc:224] Handling call
0812 08:01:28.106625 (+1607330us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:26.500947 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:26.501020 (+    73us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:26.501023 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:26.501025 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:26.501107 (+    82us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:26.501113 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:26.501116 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:26.501174 (+    58us) tablet.cc:701] Row locks acquired
0812 08:01:26.501175 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:26.501205 (+    30us) write_op.cc:270] Start()
0812 08:01:26.501224 (+    19us) write_op.cc:276] Timestamp: P: 1786521686500935 usec, L: 0
0812 08:01:26.501226 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:26.501718 (+   492us) log.cc:844] Serialized 696 byte log entry
0812 08:01:28.105160 (+1603442us) op_driver.cc:464] REPLICATION: finished
0812 08:01:28.106005 (+   845us) write_op.cc:301] APPLY: starting
0812 08:01:28.106024 (+    19us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:28.106067 (+    43us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:28.106079 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:28.106260 (+   181us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:28.106297 (+    37us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:28.106302 (+     5us) write_op.cc:312] APPLY: finished
0812 08:01:28.106376 (+    74us) log.cc:844] Serialized 77 byte log entry
0812 08:01:28.106539 (+   163us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:28.106556 (+    17us) write_op.cc:454] Released schema lock
0812 08:01:28.106562 (+     6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":792,"num_ops":10,"prepare.queue_time_us":1292,"prepare.run_cpu_time_us":496,"prepare.run_wall_time_us":821,"replication_time_us":1603901,"spinlock_wait_cycles":874240,"thread_start_us":88,"threads_started":1,"wal-append.queue_time_us":666}]]}
I20260812 08:01:28.107669 11018 rpcz_store.cc:276] 0812 08:01:26.499408 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:26.499463 (+    55us) service_pool.cc:224] Handling call
0812 08:01:28.107595 (+1608132us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:26.501790 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:26.501858 (+    68us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:26.501861 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:26.501863 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:26.504259 (+  2396us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:26.504266 (+     7us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:26.504271 (+     5us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:26.504341 (+    70us) tablet.cc:701] Row locks acquired
0812 08:01:26.504342 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:26.504371 (+    29us) write_op.cc:270] Start()
0812 08:01:26.505276 (+   905us) write_op.cc:276] Timestamp: P: 1786521686504101 usec, L: 0
0812 08:01:26.505278 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:26.505385 (+   107us) log.cc:844] Serialized 686 byte log entry
0812 08:01:28.105216 (+1599831us) op_driver.cc:464] REPLICATION: finished
0812 08:01:28.107182 (+  1966us) write_op.cc:301] APPLY: starting
0812 08:01:28.107193 (+    11us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:28.107222 (+    29us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:28.107234 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:28.107400 (+   166us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:28.107425 (+    25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:28.107430 (+     5us) write_op.cc:312] APPLY: finished
0812 08:01:28.107484 (+    54us) log.cc:844] Serialized 77 byte log entry
0812 08:01:28.107533 (+    49us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:28.107544 (+    11us) write_op.cc:454] Released schema lock
0812 08:01:28.107549 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1923,"num_ops":10,"prepare.queue_time_us":2088,"prepare.run_cpu_time_us":460,"prepare.run_wall_time_us":3646,"replication_time_us":1599925}]]}
I20260812 08:01:28.108316 11016 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2081, attempt_no=0}) took 1604 ms. Trace:
I20260812 08:01:28.108338 11018 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2080, attempt_no=0}) took 1604 ms. Trace:
I20260812 08:01:28.108392 11018 rpcz_store.cc:276] 0812 08:01:26.503521 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:26.503896 (+   375us) service_pool.cc:224] Handling call
0812 08:01:28.108330 (+1604434us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:26.505940 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:26.506010 (+    70us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:26.506014 (+     4us) write_op.cc:435] Acquired schema lock
0812 08:01:26.506016 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:26.506099 (+    83us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:26.506105 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:26.506107 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:26.506165 (+    58us) tablet.cc:701] Row locks acquired
0812 08:01:26.506166 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:26.506194 (+    28us) write_op.cc:270] Start()
0812 08:01:26.506215 (+    21us) write_op.cc:276] Timestamp: P: 1786521686505925 usec, L: 0
0812 08:01:26.506217 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:26.507184 (+   967us) log.cc:844] Serialized 696 byte log entry
0812 08:01:28.105293 (+1598109us) op_driver.cc:464] REPLICATION: finished
0812 08:01:28.107964 (+  2671us) write_op.cc:301] APPLY: starting
0812 08:01:28.107972 (+     8us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:28.107999 (+    27us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:28.108010 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:28.108169 (+   159us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:28.108195 (+    26us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:28.108200 (+     5us) write_op.cc:312] APPLY: finished
0812 08:01:28.108249 (+    49us) log.cc:844] Serialized 77 byte log entry
0812 08:01:28.108276 (+    27us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:28.108286 (+    10us) write_op.cc:454] Released schema lock
0812 08:01:28.108290 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2652,"num_ops":10,"prepare.queue_time_us":1815,"prepare.run_cpu_time_us":436,"prepare.run_wall_time_us":1291,"replication_time_us":1599066}]]}
I20260812 08:01:28.109135 11018 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2082, attempt_no=0}) took 1604 ms. Trace:
I20260812 08:01:28.109187 11018 rpcz_store.cc:276] 0812 08:01:26.504356 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:26.504392 (+    36us) service_pool.cc:224] Handling call
0812 08:01:28.109125 (+1604733us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:26.507257 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:26.507325 (+    68us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:26.507329 (+     4us) write_op.cc:435] Acquired schema lock
0812 08:01:26.507330 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:26.507412 (+    82us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:26.507418 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:26.507421 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:26.507496 (+    75us) tablet.cc:701] Row locks acquired
0812 08:01:26.507498 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:26.507527 (+    29us) write_op.cc:270] Start()
0812 08:01:26.507545 (+    18us) write_op.cc:276] Timestamp: P: 1786521686507256 usec, L: 0
0812 08:01:26.507547 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:26.507648 (+   101us) log.cc:844] Serialized 696 byte log entry
0812 08:01:28.105318 (+1597670us) op_driver.cc:464] REPLICATION: finished
0812 08:01:28.108731 (+  3413us) write_op.cc:301] APPLY: starting
0812 08:01:28.108740 (+     9us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:28.108764 (+    24us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:28.108773 (+     9us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:28.108953 (+   180us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:28.108978 (+    25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:28.108982 (+     4us) write_op.cc:312] APPLY: finished
0812 08:01:28.109031 (+    49us) log.cc:844] Serialized 77 byte log entry
0812 08:01:28.109063 (+    32us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:28.109073 (+    10us) write_op.cc:454] Released schema lock
0812 08:01:28.109077 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3393,"num_ops":10,"prepare.queue_time_us":2621,"prepare.run_cpu_time_us":441,"prepare.run_wall_time_us":441,"replication_time_us":1597761}]]}
I20260812 08:01:28.109726  9809 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 14 FOLLOWER]: Rejecting Update request from peer 3ae3f7a6e9f943e996a6dcea504878d0 for earlier term 13. Current term is 14. Ops: []
I20260812 08:01:28.110378 10738 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 14 LEADER]: Rejecting Update request from peer 3ae3f7a6e9f943e996a6dcea504878d0 for earlier term 13. Current term is 14. Ops: [13.2094-13.2096]
I20260812 08:01:28.111166  8118 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, faafbc72cbd444ff90977a2cdb091b72
W20260812 08:01:28.111225 10411 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
I20260812 08:01:28.111303  8225 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 14 FOLLOWER]: Rejecting Update request from peer 3ae3f7a6e9f943e996a6dcea504878d0 for earlier term 13. Current term is 14. Ops: [13.2045-13.2096]
I20260812 08:01:28.111459 11089 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260812 08:01:28.108374 11016 rpcz_store.cc:276] 0812 08:01:26.503584 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:26.503620 (+    36us) service_pool.cc:224] Handling call
0812 08:01:28.108306 (+1604686us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:26.505475 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:26.505548 (+    73us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:26.505551 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:26.505553 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:26.505636 (+    83us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:26.505642 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:26.505645 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:26.505703 (+    58us) tablet.cc:701] Row locks acquired
0812 08:01:26.505705 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:26.505734 (+    29us) write_op.cc:270] Start()
0812 08:01:26.505751 (+    17us) write_op.cc:276] Timestamp: P: 1786521686505464 usec, L: 0
0812 08:01:26.505753 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:26.505856 (+   103us) log.cc:844] Serialized 696 byte log entry
0812 08:01:28.105263 (+1599407us) op_driver.cc:464] REPLICATION: finished
0812 08:01:28.107919 (+  2656us) write_op.cc:301] APPLY: starting
0812 08:01:28.107931 (+    12us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:28.107958 (+    27us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:28.107969 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:28.108124 (+   155us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:28.108153 (+    29us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:28.108157 (+     4us) write_op.cc:312] APPLY: finished
0812 08:01:28.108208 (+    51us) log.cc:844] Serialized 77 byte log entry
0812 08:01:28.108242 (+    34us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:28.108253 (+    11us) write_op.cc:454] Released schema lock
0812 08:01:28.108258 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2632,"num_ops":10,"prepare.queue_time_us":1587,"prepare.run_cpu_time_us":432,"prepare.run_wall_time_us":431,"replication_time_us":1599481}]]}
W20260812 08:01:28.112622 10413 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:28.113150 10411 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:28.112622 10412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:28.114089 10412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
I20260812 08:01:28.114183  8004 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 14 FOLLOWER]: Rejecting Update request from peer 3ae3f7a6e9f943e996a6dcea504878d0 for earlier term 13. Current term is 14. Ops: [13.1968-13.2096]
I20260812 08:01:28.115576  7381 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 14 FOLLOWER]: Rejecting Update request from peer 3ae3f7a6e9f943e996a6dcea504878d0 for earlier term 13. Current term is 14. Ops: [13.2007-13.2096]
I20260812 08:01:28.118786 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 14 candidate_status { last_received { term: 13 index: 2093 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0"
I20260812 08:01:28.118754 10465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 14 candidate_status { last_received { term: 13 index: 2093 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:28.118942 10460 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 56b60424ed9f4a079689ad82a318d29a for term 14 because replica has last-logged OpId of term: 13 index: 2096, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 2093.
W20260812 08:01:28.121474 10004 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Illegal state: replica faafbc72cbd444ff90977a2cdb091b72 is not leader of this config: current role FOLLOWER
W20260812 08:01:28.121479 10006 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Illegal state: replica faafbc72cbd444ff90977a2cdb091b72 is not leader of this config: current role FOLLOWER
W20260812 08:01:28.121733 10007 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Illegal state: replica faafbc72cbd444ff90977a2cdb091b72 is not leader of this config: current role FOLLOWER
W20260812 08:01:28.122126 10010 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Illegal state: replica faafbc72cbd444ff90977a2cdb091b72 is not leader of this config: current role FOLLOWER
W20260812 08:01:28.122655 10009 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Illegal state: replica faafbc72cbd444ff90977a2cdb091b72 is not leader of this config: current role FOLLOWER
W20260812 08:01:28.126153  9751 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48992: Illegal state: replica 1664e32d3cd0473592085aa522379f22 is not leader of this config: current role FOLLOWER
W20260812 08:01:28.128286  9751 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48992: Illegal state: replica 1664e32d3cd0473592085aa522379f22 is not leader of this config: current role FOLLOWER
W20260812 08:01:28.128365  9748 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48992: Illegal state: replica 1664e32d3cd0473592085aa522379f22 is not leader of this config: current role FOLLOWER
W20260812 08:01:28.128365  9744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48992: Illegal state: replica 1664e32d3cd0473592085aa522379f22 is not leader of this config: current role FOLLOWER
W20260812 08:01:28.130308  9748 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48992: Illegal state: replica 1664e32d3cd0473592085aa522379f22 is not leader of this config: current role FOLLOWER
I20260812 08:01:28.137951  7381 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 14 FOLLOWER]: Refusing update from remote peer 56b60424ed9f4a079689ad82a318d29a: Log matching property violated. Preceding OpId in replica: term: 13 index: 2006. Preceding OpId from leader: term: 14 index: 2095. (index mismatch)
I20260812 08:01:28.138005  8225 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 14 FOLLOWER]: Refusing update from remote peer 56b60424ed9f4a079689ad82a318d29a: Log matching property violated. Preceding OpId in replica: term: 13 index: 2044. Preceding OpId from leader: term: 14 index: 2095. (index mismatch)
I20260812 08:01:28.138685 10079 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 14 FOLLOWER]: Refusing update from remote peer 56b60424ed9f4a079689ad82a318d29a: Log matching property violated. Preceding OpId in replica: term: 13 index: 2096. Preceding OpId from leader: term: 14 index: 2095. (term mismatch)
I20260812 08:01:28.138764 10079 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting all ops after (but not including) 2094
I20260812 08:01:28.138685 11090 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2094, Last known committed idx: 2043, Time since last communication: 0.000s
I20260812 08:01:28.138792 10079 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 13.2095
I20260812 08:01:28.138836  8004 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 14 FOLLOWER]: Refusing update from remote peer 56b60424ed9f4a079689ad82a318d29a: Log matching property violated. Preceding OpId in replica: term: 13 index: 1967. Preceding OpId from leader: term: 14 index: 2095. (index mismatch)
I20260812 08:01:28.139003 10079 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 13.2096
I20260812 08:01:28.139156 11134 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Connected to new peer: Peer: permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2094, Last known committed idx: 1999, Time since last communication: 0.000s
I20260812 08:01:28.140093 11090 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Connected to new peer: Peer: permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2094, Last known committed idx: 2088, Time since last communication: 0.000s
I20260812 08:01:28.140200 11090 consensus_queue.cc:1243] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Peer faafbc72cbd444ff90977a2cdb091b72 log is divergent from this leader: its last log entry 13.2094 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2088
I20260812 08:01:28.139552  9809 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 14 FOLLOWER]: Refusing update from remote peer 56b60424ed9f4a079689ad82a318d29a: Log matching property violated. Preceding OpId in replica: term: 13 index: 2096. Preceding OpId from leader: term: 14 index: 2095. (term mismatch)
I20260812 08:01:28.140435  9809 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting all ops after (but not including) 2094
I20260812 08:01:28.140484  9809 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting uncommitted WRITE_OP operation due to leader change: 13.2095
I20260812 08:01:28.140456 11090 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Connected to new peer: Peer: permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2094, Last known committed idx: 1960, Time since last communication: 0.000s
I20260812 08:01:28.140718  9809 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting uncommitted WRITE_OP operation due to leader change: 13.2096
I20260812 08:01:28.141678 10079 raft_consensus.cc:1217] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 14 FOLLOWER]: Deduplicated request from leader. Original: 13.2088->[13.2089-14.2098]   Dedup: 13.2093->[14.2094-14.2098]
I20260812 08:01:28.141803 10079 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting all ops after (but not including) 2093
I20260812 08:01:28.141830 10079 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 13.2094
I20260812 08:01:28.142836 10460 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 14 FOLLOWER]: Refusing update from remote peer 56b60424ed9f4a079689ad82a318d29a: Log matching property violated. Preceding OpId in replica: term: 13 index: 2096. Preceding OpId from leader: term: 14 index: 2095. (term mismatch)
I20260812 08:01:28.142920 10460 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting all ops after (but not including) 2094
I20260812 08:01:28.142995 10460 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 13.2095
W20260812 08:01:28.143144 10460 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Aborted: Op aborted by new leader
I20260812 08:01:28.143291 10460 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2083, attempt_no=0}) took 1631 ms. Trace:
I20260812 08:01:28.143334 10460 rpcz_store.cc:276] 0812 08:01:26.511461 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:26.511950 (+   489us) service_pool.cc:224] Handling call
0812 08:01:28.143280 (+1631330us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:26.513440 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:26.513517 (+    77us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:26.513520 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:26.513522 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:26.513601 (+    79us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:26.513606 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:26.514089 (+   483us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:26.514153 (+    64us) tablet.cc:701] Row locks acquired
0812 08:01:26.514154 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:26.514183 (+    29us) write_op.cc:270] Start()
0812 08:01:26.514202 (+    19us) write_op.cc:276] Timestamp: P: 1786521686513912 usec, L: 0
0812 08:01:26.514204 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:26.514526 (+   322us) log.cc:844] Serialized 686 byte log entry
0812 08:01:28.143085 (+1628559us) op_driver.cc:464] REPLICATION: finished
0812 08:01:28.143097 (+    12us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 08:01:28.143101 (+     4us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:28.143130 (+    29us) write_op.cc:454] Released schema lock
0812 08:01:28.143138 (+     8us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":1274,"prepare.run_cpu_time_us":486,"prepare.run_wall_time_us":1366,"raft.queue_time_us":24,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":155,"replication_time_us":1628825}]]}
I20260812 08:01:28.143668 10460 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 13.2096
W20260812 08:01:28.143783 10460 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Aborted: Op aborted by new leader
I20260812 08:01:28.143894 10460 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2085, attempt_no=0}) took 1632 ms. Trace:
I20260812 08:01:28.143935 10460 rpcz_store.cc:276] 0812 08:01:26.511580 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:26.511986 (+   406us) service_pool.cc:224] Handling call
0812 08:01:28.143885 (+1631899us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:26.514826 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:26.514887 (+    61us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:26.514890 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:26.514891 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:26.515783 (+   892us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:26.515790 (+     7us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:26.515793 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:26.515857 (+    64us) tablet.cc:701] Row locks acquired
0812 08:01:26.515859 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:26.515886 (+    27us) write_op.cc:270] Start()
0812 08:01:26.515906 (+    20us) write_op.cc:276] Timestamp: P: 1786521686515617 usec, L: 0
0812 08:01:26.515908 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:26.516029 (+   121us) log.cc:844] Serialized 696 byte log entry
0812 08:01:28.143747 (+1627718us) op_driver.cc:464] REPLICATION: finished
0812 08:01:28.143753 (+     6us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 08:01:28.143756 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:28.143774 (+    18us) write_op.cc:454] Released schema lock
0812 08:01:28.143780 (+     6us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":2600,"prepare.run_cpu_time_us":465,"prepare.run_wall_time_us":1270,"raft.queue_time_us":26,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":138,"replication_time_us":1627807,"spinlock_wait_cycles":48512}]]}
I20260812 08:01:28.145181 11090 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2094, Last known committed idx: 2088, Time since last communication: 0.000s
I20260812 08:01:28.145294 11090 consensus_queue.cc:1243] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Peer 1664e32d3cd0473592085aa522379f22 log is divergent from this leader: its last log entry 13.2094 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2088
I20260812 08:01:28.145498 11115 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2094, Last known committed idx: 2093, Time since last communication: 0.000s
I20260812 08:01:28.145598 11115 consensus_queue.cc:1243] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Peer 3ae3f7a6e9f943e996a6dcea504878d0 log is divergent from this leader: its last log entry 13.2094 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2093
I20260812 08:01:28.146356  9809 raft_consensus.cc:1217] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 14 FOLLOWER]: Deduplicated request from leader. Original: 13.2088->[13.2089-14.2099]   Dedup: 13.2093->[14.2094-14.2099]
I20260812 08:01:28.146486  9809 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting all ops after (but not including) 2093
I20260812 08:01:28.146517  9809 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting uncommitted WRITE_OP operation due to leader change: 13.2094
I20260812 08:01:28.154479 10460 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting all ops after (but not including) 2093
I20260812 08:01:28.154567 10460 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 13.2094
W20260812 08:01:28.154721 10460 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Aborted: Op aborted by new leader
I20260812 08:01:28.154867 10460 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2084, attempt_no=0}) took 1643 ms. Trace:
I20260812 08:01:28.154914 10460 rpcz_store.cc:276] 0812 08:01:26.511530 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:26.511600 (+    70us) service_pool.cc:224] Handling call
0812 08:01:28.154855 (+1643255us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:26.512387 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:26.512479 (+    92us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:26.512482 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:26.512483 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:26.512568 (+    85us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:26.512573 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:26.512577 (+     4us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:26.512636 (+    59us) tablet.cc:701] Row locks acquired
0812 08:01:26.512637 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:26.512664 (+    27us) write_op.cc:270] Start()
0812 08:01:26.512681 (+    17us) write_op.cc:276] Timestamp: P: 1786521686512394 usec, L: 0
0812 08:01:26.512683 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:26.512787 (+   104us) log.cc:844] Serialized 696 byte log entry
0812 08:01:28.154649 (+1641862us) op_driver.cc:464] REPLICATION: finished
0812 08:01:28.154658 (+     9us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 08:01:28.154662 (+     4us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:28.154707 (+    45us) write_op.cc:454] Released schema lock
0812 08:01:28.154714 (+     7us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":457,"prepare.run_cpu_time_us":514,"prepare.run_wall_time_us":1034,"raft.queue_time_us":61,"raft.run_cpu_time_us":314,"raft.run_wall_time_us":313,"replication_time_us":1641919}]]}
I20260812 08:01:30.035933 11195 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:30.036115 11195 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:30.037145 11195 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:30.037372  9808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 15 candidate_status { last_received { term: 14 index: 2188 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:30.037418 10465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 15 candidate_status { last_received { term: 14 index: 2188 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:30.037523  8220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 15 candidate_status { last_received { term: 14 index: 2188 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:30.037617  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 15 candidate_status { last_received { term: 14 index: 2188 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:30.037899 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 15 candidate_status { last_received { term: 14 index: 2188 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:30.038115  7288 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 16901c5559a748a9aa770d73e254099c; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df
I20260812 08:01:30.038292 11195 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260812 08:01:30.257788 11197 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:30.257938 11197 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:30.258781 11197 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:30.258941 10465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 15 candidate_status { last_received { term: 14 index: 2272 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:30.259068 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 15 candidate_status { last_received { term: 14 index: 2272 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:30.259153  8220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 15 candidate_status { last_received { term: 14 index: 2272 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:30.259460  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 15 candidate_status { last_received { term: 14 index: 2272 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:30.259613  7381 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 14.
I20260812 08:01:30.259888  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 15 candidate_status { last_received { term: 14 index: 2272 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:30.260286  9706 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c; no voters: 3ae3f7a6e9f943e996a6dcea504878d0, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:30.260457 11197 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260812 08:01:30.261178 11198 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:30.261274 11198 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:30.261844 11199 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:30.261922 11198 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:30.262135  9808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 15 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:30.262121 11199 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:30.262243 10465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 15 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:30.262274  9808 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 14.
I20260812 08:01:30.262329 10465 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 14.
I20260812 08:01:30.262681  8220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 15 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:30.262759  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 15 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:30.262853 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 15 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:30.262969 10079 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 14.
I20260812 08:01:30.263003  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 15 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:30.263134  7381 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 14.
I20260812 08:01:30.263376  9969 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, faafbc72cbd444ff90977a2cdb091b72; no voters: 6801d45ca6614156906e5d571ebcb8a6
I20260812 08:01:30.263489  9808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 15 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:30.263590 11198 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260812 08:01:30.263602  9808 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 14.
I20260812 08:01:30.263653 11198 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:30.263702 11198 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 14 FOLLOWER]: Advancing to term 15
I20260812 08:01:30.264024 11199 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:30.264158  8220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 15 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:30.264219  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 15 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:30.264490  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 15 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:30.264617  7381 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 14.
I20260812 08:01:30.264879 10367 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, faafbc72cbd444ff90977a2cdb091b72; no voters: 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df
I20260812 08:01:30.265102 11198 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:30.265164 11199 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260812 08:01:30.265216 11199 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:30.265254 11199 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 14 FOLLOWER]: Advancing to term 15
I20260812 08:01:30.265872 10465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 15 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0"
I20260812 08:01:30.265887  9808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 15 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22"
I20260812 08:01:30.265978  9808 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 14 FOLLOWER]: Advancing to term 15
I20260812 08:01:30.266402 11199 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:30.266641 10465 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate faafbc72cbd444ff90977a2cdb091b72 in current term 15: Already voted for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in this term.
I20260812 08:01:30.267050  9808 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 15.
I20260812 08:01:30.267319 11199 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 15 election: Requested vote from peers faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:30.267410  8220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 15 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
I20260812 08:01:30.267791 11198 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 15 election: Requested vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:30.268011  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 15 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:01:30.268077  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 15 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:01:30.268098  8220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 15 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
I20260812 08:01:30.268817  9808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 15 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22"
I20260812 08:01:30.268978  9808 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ae3f7a6e9f943e996a6dcea504878d0 in current term 15: Already voted for candidate faafbc72cbd444ff90977a2cdb091b72 in this term.
I20260812 08:01:30.269042 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 15 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72"
I20260812 08:01:30.269165 10079 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ae3f7a6e9f943e996a6dcea504878d0 in current term 15: Already voted for candidate faafbc72cbd444ff90977a2cdb091b72 in this term.
I20260812 08:01:30.269389 10363 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 3ae3f7a6e9f943e996a6dcea504878d0; no voters: 1664e32d3cd0473592085aa522379f22, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:30.269559 11199 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260812 08:01:30.273006  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 15 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:01:30.273149  7381 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 14 FOLLOWER]: Advancing to term 15
I20260812 08:01:30.273389  7375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 15 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:01:30.274242  7381 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 15.
I20260812 08:01:30.602283 11200 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:30.602430 11200 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:30.603513 11200 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183)
I20260812 08:01:30.603500 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 15 candidate_status { last_received { term: 14 index: 2145 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:30.603684 10079 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df in current term 15: Already voted for candidate faafbc72cbd444ff90977a2cdb091b72 in this term.
I20260812 08:01:30.603684 10465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 15 candidate_status { last_received { term: 14 index: 2145 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:30.603773  8220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 15 candidate_status { last_received { term: 14 index: 2145 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:30.603785  9808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 15 candidate_status { last_received { term: 14 index: 2145 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:30.603843 10465 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df in current term 15: Already voted for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in this term.
I20260812 08:01:30.603932  9808 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df in current term 15: Already voted for candidate faafbc72cbd444ff90977a2cdb091b72 in this term.
I20260812 08:01:30.604322  7898 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:30.604352  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 15 candidate_status { last_received { term: 14 index: 2145 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:30.604490  7381 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df in current term 15: Already voted for candidate faafbc72cbd444ff90977a2cdb091b72 in this term.
I20260812 08:01:30.604506 11200 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 14 FOLLOWER]: Advancing to term 15
I20260812 08:01:30.605479 11200 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260812 08:01:30.619411 11201 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:30.619536 11201 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:30.620239 11201 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:30.620436 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 2226 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:30.620569 10465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 2226 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:30.620630 10079 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6801d45ca6614156906e5d571ebcb8a6 in current term 15: Already voted for candidate faafbc72cbd444ff90977a2cdb091b72 in this term.
I20260812 08:01:30.620724 10465 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6801d45ca6614156906e5d571ebcb8a6 in current term 15: Already voted for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in this term.
I20260812 08:01:30.620711  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 2226 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:30.620864  7381 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6801d45ca6614156906e5d571ebcb8a6 in current term 15: Already voted for candidate faafbc72cbd444ff90977a2cdb091b72 in this term.
I20260812 08:01:30.621201  9808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 2226 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:30.621238  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 2226 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:30.621371  9808 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6801d45ca6614156906e5d571ebcb8a6 in current term 15: Already voted for candidate faafbc72cbd444ff90977a2cdb091b72 in this term.
I20260812 08:01:30.621363  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6801d45ca6614156906e5d571ebcb8a6 in term 15.
I20260812 08:01:30.621642  8116 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:30.621819 11201 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 14 FOLLOWER]: Advancing to term 15
I20260812 08:01:30.622705 11201 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260812 08:01:31.688081  7286 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.142.130:38363 timed out after 1.650s (SENT) [suppressed 1 similar messages]
W20260812 08:01:31.781133  9966 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.142.130:38363 timed out after 1.518s (SENT)
W20260812 08:01:31.784569 10364 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.142.130:38363 timed out after 1.520s (SENT)
I20260812 08:01:31.885617 11208 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:01:31.885746 11208 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:31.886375 11208 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:31.886631 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 16 candidate_status { last_received { term: 14 index: 2188 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:31.886735  9808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 16 candidate_status { last_received { term: 14 index: 2188 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:31.886821 10079 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16901c5559a748a9aa770d73e254099c for term 16 because replica has last-logged OpId of term: 14 index: 2273, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 2188.
I20260812 08:01:31.886780 10465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 16 candidate_status { last_received { term: 14 index: 2188 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:31.886854  8220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 16 candidate_status { last_received { term: 14 index: 2188 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:31.886904  9808 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16901c5559a748a9aa770d73e254099c for term 16 because replica has last-logged OpId of term: 14 index: 2272, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 2188.
I20260812 08:01:31.886929 10465 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16901c5559a748a9aa770d73e254099c for term 16 because replica has last-logged OpId of term: 14 index: 2273, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 2188.
I20260812 08:01:31.886996  8220 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16901c5559a748a9aa770d73e254099c for term 16 because replica has last-logged OpId of term: 14 index: 2226, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 2188.
I20260812 08:01:31.887380  7287 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 16901c5559a748a9aa770d73e254099c; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:31.887521  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 16 candidate_status { last_received { term: 14 index: 2188 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:31.887588 11208 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260812 08:01:31.887681  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16901c5559a748a9aa770d73e254099c in term 15.
I20260812 08:01:32.155898 11209 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:01:32.156080 11209 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:32.157073 11209 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:32.157321 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 16 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:32.157374  8220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 16 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:32.157426  9808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 16 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:32.157518 10079 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 15.
I20260812 08:01:32.157557  9808 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 15.
I20260812 08:01:32.157545  8220 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 15.
I20260812 08:01:32.157655  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 16 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:32.157794  7381 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 15.
I20260812 08:01:32.157991 10363 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72; no voters: 
I20260812 08:01:32.158200 11209 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260812 08:01:32.158213  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 16 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:32.158263 11209 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:01:32.158303 11209 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 15 FOLLOWER]: Advancing to term 16
I20260812 08:01:32.158334  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 15.
I20260812 08:01:32.159277 11209 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:32.160221  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 16 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:01:32.160220  8220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 16 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
I20260812 08:01:32.160348  8220 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 15 FOLLOWER]: Advancing to term 16
I20260812 08:01:32.160347  7381 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 15 FOLLOWER]: Advancing to term 16
I20260812 08:01:32.160398 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 16 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72"
I20260812 08:01:32.160531 10079 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 15 FOLLOWER]: Advancing to term 16
I20260812 08:01:32.160501  9808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 16 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22"
I20260812 08:01:32.160620  9808 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 15 FOLLOWER]: Advancing to term 16
I20260812 08:01:32.161477  8220 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 16.
I20260812 08:01:32.161479  7381 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 16.
I20260812 08:01:32.161655 10079 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 16.
I20260812 08:01:32.161733  9808 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 16.
I20260812 08:01:32.162072 10363 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72; no voters: 
I20260812 08:01:32.162137 11209 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 16 election: Requested vote from peers faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:32.162322 11209 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 16 FOLLOWER]: Leader election won for term 16
I20260812 08:01:32.162496 11209 raft_consensus.cc:697] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 16 LEADER]: Becoming Leader. State: Replica: 3ae3f7a6e9f943e996a6dcea504878d0, State: Running, Role: LEADER
I20260812 08:01:32.162675 11209 consensus_queue.cc:237] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2265, Committed index: 2265, Last appended: 14.2273, Last appended by leader: 2273, Current term: 16, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:32.162714  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 16 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:01:32.162839  8004 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 15 FOLLOWER]: Advancing to term 16
I20260812 08:01:32.163746  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 16.
I20260812 08:01:32.165683  6748 catalog_manager.cc:5719] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 reported cstate change: term changed from 14 to 16, leader changed from 56b60424ed9f4a079689ad82a318d29a (127.5.142.130) to 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129). New cstate: current_term: 16 leader_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } health_report { overall_health: UNKNOWN } } }
W20260812 08:01:32.203778  9707 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.142.130:38363 timed out after 1.944s (SENT) [suppressed 1 similar messages]
W20260812 08:01:32.453842  7902 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.142.130:38363 timed out after 1.849s (SENT) [suppressed 1 similar messages]
W20260812 08:01:32.486092 10623 connection.cc:256] RPC call timeout handler was delayed by -3.1753698012325913s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 08:01:32.486919 10728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 15 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a"
W20260812 08:01:32.487248 10624 connection.cc:256] RPC call timeout handler was delayed by -2.8312237011268735s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 08:01:32.487279 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 15 candidate_status { last_received { term: 14 index: 2188 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:32.487509 10730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 15 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a"
I20260812 08:01:32.487735 10725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 16 candidate_status { last_received { term: 14 index: 2188 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
W20260812 08:01:32.487778 10631 connection.cc:256] RPC call timeout handler was delayed by -3.0530053218826652s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 08:01:32.487963 10726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 15 candidate_status { last_received { term: 14 index: 2188 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:32.488197 10729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 15 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
W20260812 08:01:32.487717 10629 connection.cc:256] RPC call timeout handler was delayed by -2.9414214920252562s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 08:01:32.490680 10719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 2226 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:32.490828 11175 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:58954 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2254, attempt_no=0}) took 3740 ms. Trace:
I20260812 08:01:32.491084  8220 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 16 FOLLOWER]: Rejecting Update request from peer 56b60424ed9f4a079689ad82a318d29a for earlier term 14. Current term is 16. Ops: [14.2227-14.2273]
I20260812 08:01:32.491374 10724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 16 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a"
I20260812 08:01:32.490909 11175 rpcz_store.cc:276] 0812 08:01:28.750573 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:28.750617 (+    44us) service_pool.cc:224] Handling call
0812 08:01:32.490815 (+3740198us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:28.750977 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:28.751073 (+    96us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:28.751076 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:28.751078 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:28.751173 (+    95us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:28.751191 (+    18us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:28.751194 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:28.751264 (+    70us) tablet.cc:701] Row locks acquired
0812 08:01:28.751266 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:28.751300 (+    34us) write_op.cc:270] Start()
0812 08:01:28.751323 (+    23us) write_op.cc:276] Timestamp: P: 1786521688751260 usec, L: 0
0812 08:01:28.751325 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:28.751449 (+   124us) log.cc:844] Serialized 696 byte log entry
0812 08:01:32.489652 (+3738203us) op_driver.cc:464] REPLICATION: finished
0812 08:01:32.490193 (+   541us) write_op.cc:301] APPLY: starting
0812 08:01:32.490212 (+    19us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:32.490256 (+    44us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:32.490268 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:32.490455 (+   187us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:32.490484 (+    29us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:32.490489 (+     5us) write_op.cc:312] APPLY: finished
0812 08:01:32.490562 (+    73us) log.cc:844] Serialized 77 byte log entry
0812 08:01:32.490718 (+   156us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:32.490739 (+    21us) write_op.cc:454] Released schema lock
0812 08:01:32.490745 (+     6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":501,"num_ops":10,"prepare.queue_time_us":43,"prepare.run_cpu_time_us":539,"prepare.run_wall_time_us":743,"raft.queue_time_us":30,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":149,"replication_time_us":3738293,"thread_start_us":93,"threads_started":1}]]}
I20260812 08:01:32.491931 10738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 15 candidate_status { last_received { term: 14 index: 2145 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:32.492125 10734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 15 candidate_status { last_received { term: 14 index: 2145 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:32.492224 11175 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:58954 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2255, attempt_no=0}) took 3740 ms. Trace:
I20260812 08:01:32.492309 10723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 15 candidate_status { last_received { term: 14 index: 2272 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:32.492720 10732 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 15 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:32.492277 11175 rpcz_store.cc:276] 0812 08:01:28.751811 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:28.751846 (+    35us) service_pool.cc:224] Handling call
0812 08:01:32.492215 (+3740369us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:28.752157 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:28.752244 (+    87us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:28.752248 (+     4us) write_op.cc:435] Acquired schema lock
0812 08:01:28.752249 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:28.752334 (+    85us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:28.752340 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:28.752343 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:28.752406 (+    63us) tablet.cc:701] Row locks acquired
0812 08:01:28.752408 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:28.752432 (+    24us) write_op.cc:270] Start()
0812 08:01:28.752451 (+    19us) write_op.cc:276] Timestamp: P: 1786521688752395 usec, L: 0
0812 08:01:28.752453 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:28.752555 (+   102us) log.cc:844] Serialized 696 byte log entry
0812 08:01:32.489697 (+3737142us) op_driver.cc:464] REPLICATION: finished
0812 08:01:32.491808 (+  2111us) write_op.cc:301] APPLY: starting
0812 08:01:32.491818 (+    10us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:32.491848 (+    30us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:32.491860 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:32.492026 (+   166us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:32.492052 (+    26us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:32.492057 (+     5us) write_op.cc:312] APPLY: finished
0812 08:01:32.492110 (+    53us) log.cc:844] Serialized 77 byte log entry
0812 08:01:32.492141 (+    31us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:32.492161 (+    20us) write_op.cc:454] Released schema lock
0812 08:01:32.492166 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2090,"num_ops":10,"prepare.queue_time_us":42,"prepare.run_cpu_time_us":437,"prepare.run_wall_time_us":436,"replication_time_us":3737235,"thread_start_us":105,"threads_started":1}]]}
I20260812 08:01:32.493151 10729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 15 candidate_status { last_received { term: 14 index: 2272 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:32.493379 10465 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 16 LEADER]: Rejecting Update request from peer 56b60424ed9f4a079689ad82a318d29a for earlier term 14. Current term is 16. Ops: []
I20260812 08:01:32.493567 11175 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:58954 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2257, attempt_no=0}) took 3741 ms. Trace:
I20260812 08:01:32.493618 11175 rpcz_store.cc:276] 0812 08:01:28.752592 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:28.752624 (+    32us) service_pool.cc:224] Handling call
0812 08:01:32.493559 (+3740935us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:28.753161 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:28.753232 (+    71us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:28.753235 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:28.753237 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:28.753314 (+    77us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:28.753320 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:28.753323 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:28.753384 (+    61us) tablet.cc:701] Row locks acquired
0812 08:01:28.753385 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:28.753435 (+    50us) write_op.cc:270] Start()
0812 08:01:28.753453 (+    18us) write_op.cc:276] Timestamp: P: 1786521688753397 usec, L: 0
0812 08:01:28.753455 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:28.753550 (+    95us) log.cc:844] Serialized 696 byte log entry
0812 08:01:32.489963 (+3736413us) op_driver.cc:464] REPLICATION: finished
0812 08:01:32.493171 (+  3208us) write_op.cc:301] APPLY: starting
0812 08:01:32.493180 (+     9us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:32.493207 (+    27us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:32.493217 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:32.493390 (+   173us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:32.493414 (+    24us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:32.493418 (+     4us) write_op.cc:312] APPLY: finished
0812 08:01:32.493474 (+    56us) log.cc:844] Serialized 77 byte log entry
0812 08:01:32.493502 (+    28us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:32.493513 (+    11us) write_op.cc:454] Released schema lock
0812 08:01:32.493517 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3173,"num_ops":10,"prepare.queue_time_us":302,"prepare.run_cpu_time_us":423,"prepare.run_wall_time_us":422,"replication_time_us":3736493,"spinlock_wait_cycles":33024,"thread_start_us":60,"threads_started":1}]]}
I20260812 08:01:32.494194 10725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 15 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:32.494287 11175 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:58954 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2258, attempt_no=0}) took 3741 ms. Trace:
I20260812 08:01:32.494438 10730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 15 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a"
I20260812 08:01:32.494346 11175 rpcz_store.cc:276] 0812 08:01:28.752971 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:28.753002 (+    31us) service_pool.cc:224] Handling call
0812 08:01:32.494278 (+3741276us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:28.753610 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:28.753690 (+    80us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:28.753694 (+     4us) write_op.cc:435] Acquired schema lock
0812 08:01:28.753695 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:28.753775 (+    80us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:28.753780 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:28.753783 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:28.753845 (+    62us) tablet.cc:701] Row locks acquired
0812 08:01:28.753847 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:28.753869 (+    22us) write_op.cc:270] Start()
0812 08:01:28.753887 (+    18us) write_op.cc:276] Timestamp: P: 1786521688753833 usec, L: 0
0812 08:01:28.753889 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:28.753982 (+    93us) log.cc:844] Serialized 686 byte log entry
0812 08:01:32.490060 (+3736078us) op_driver.cc:464] REPLICATION: finished
0812 08:01:32.493907 (+  3847us) write_op.cc:301] APPLY: starting
0812 08:01:32.493914 (+     7us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:32.493939 (+    25us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:32.493950 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:32.494116 (+   166us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:32.494139 (+    23us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:32.494143 (+     4us) write_op.cc:312] APPLY: finished
0812 08:01:32.494197 (+    54us) log.cc:844] Serialized 77 byte log entry
0812 08:01:32.494225 (+    28us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:32.494235 (+    10us) write_op.cc:454] Released schema lock
0812 08:01:32.494239 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3825,"num_ops":10,"prepare.queue_time_us":381,"prepare.run_cpu_time_us":405,"prepare.run_wall_time_us":404,"replication_time_us":3736161}]]}
I20260812 08:01:32.494870 10728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 16 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:32.495458  9808 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 16 FOLLOWER]: Rejecting Update request from peer 56b60424ed9f4a079689ad82a318d29a for earlier term 14. Current term is 16. Ops: [14.2273-14.2273]
I20260812 08:01:32.494661 10726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 15 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:32.494773 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 15 candidate_status { last_received { term: 14 index: 2273 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a"
I20260812 08:01:32.496306 11175 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:58954 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2259, attempt_no=0}) took 3743 ms. Trace:
I20260812 08:01:32.496418 11175 rpcz_store.cc:276] 0812 08:01:28.753314 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:28.753346 (+    32us) service_pool.cc:224] Handling call
0812 08:01:32.496295 (+3742949us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:28.754041 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:28.754108 (+    67us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:28.754111 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:28.754112 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:28.754190 (+    78us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:28.754195 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:28.754198 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:28.754255 (+    57us) tablet.cc:701] Row locks acquired
0812 08:01:28.754257 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:28.754283 (+    26us) write_op.cc:270] Start()
0812 08:01:28.754301 (+    18us) write_op.cc:276] Timestamp: P: 1786521688754246 usec, L: 0
0812 08:01:28.754303 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:28.754414 (+   111us) log.cc:844] Serialized 696 byte log entry
0812 08:01:32.490083 (+3735669us) op_driver.cc:464] REPLICATION: finished
0812 08:01:32.495828 (+  5745us) write_op.cc:301] APPLY: starting
0812 08:01:32.495837 (+     9us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:32.495863 (+    26us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:32.495873 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:32.496045 (+   172us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:32.496069 (+    24us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:32.496073 (+     4us) write_op.cc:312] APPLY: finished
0812 08:01:32.496185 (+   112us) log.cc:844] Serialized 77 byte log entry
0812 08:01:32.496225 (+    40us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:32.496239 (+    14us) write_op.cc:454] Released schema lock
0812 08:01:32.496244 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":5719,"num_ops":10,"prepare.queue_time_us":485,"prepare.run_cpu_time_us":406,"prepare.run_wall_time_us":405,"replication_time_us":3735771,"spinlock_wait_cycles":16128}]]}
I20260812 08:01:32.496798  9966 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, faafbc72cbd444ff90977a2cdb091b72; no voters: 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df
I20260812 08:01:32.497488 11175 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:58954 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2260, attempt_no=0}) took 3743 ms. Trace:
I20260812 08:01:32.497720  7381 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 16 FOLLOWER]: Rejecting Update request from peer 56b60424ed9f4a079689ad82a318d29a for earlier term 14. Current term is 16. Ops: [14.2189-14.2273]
I20260812 08:01:32.497593 11175 rpcz_store.cc:276] 0812 08:01:28.753642 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:28.753675 (+    33us) service_pool.cc:224] Handling call
0812 08:01:32.497479 (+3743804us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:28.754477 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:28.754544 (+    67us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:28.754547 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:28.754549 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:28.754628 (+    79us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:28.754633 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:28.754636 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:28.754706 (+    70us) tablet.cc:701] Row locks acquired
0812 08:01:28.754707 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:28.754733 (+    26us) write_op.cc:270] Start()
0812 08:01:28.754752 (+    19us) write_op.cc:276] Timestamp: P: 1786521688754696 usec, L: 0
0812 08:01:28.754753 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:01:28.754850 (+    97us) log.cc:844] Serialized 686 byte log entry
0812 08:01:32.490105 (+3735255us) op_driver.cc:464] REPLICATION: finished
0812 08:01:32.497103 (+  6998us) write_op.cc:301] APPLY: starting
0812 08:01:32.497112 (+     9us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:32.497138 (+    26us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:32.497148 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:32.497320 (+   172us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:32.497344 (+    24us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:32.497348 (+     4us) write_op.cc:312] APPLY: finished
0812 08:01:32.497394 (+    46us) log.cc:844] Serialized 77 byte log entry
0812 08:01:32.497422 (+    28us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:32.497432 (+    10us) write_op.cc:454] Released schema lock
0812 08:01:32.497436 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":6977,"num_ops":10,"prepare.queue_time_us":560,"prepare.run_cpu_time_us":416,"prepare.run_wall_time_us":416,"replication_time_us":3735343}]]}
I20260812 08:01:32.498502 11237 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260812 08:01:32.494293 11226 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 }, Status: INVALID_TERM, Last received: 14.2226, Next index: 2227, Last known committed idx: 2219, Time since last communication: 0.004s
I20260812 08:01:32.493161 11235 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:58954 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2256, attempt_no=0}) took 3740 ms. Trace:
I20260812 08:01:32.498837 11227 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 }, Status: INVALID_TERM, Last received: 14.2273, Next index: 2274, Last known committed idx: 2265, Time since last communication: 0.000s
I20260812 08:01:32.499037 11227 consensus_queue.cc:1243] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Peer 3ae3f7a6e9f943e996a6dcea504878d0 log is divergent from this leader: its last log entry 16.2274 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2265
I20260812 08:01:32.499053 10079 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 16 FOLLOWER]: Rejecting Update request from peer 56b60424ed9f4a079689ad82a318d29a for earlier term 14. Current term is 16. Ops: []
I20260812 08:01:32.499226  8004 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 16 FOLLOWER]: Rejecting Update request from peer 56b60424ed9f4a079689ad82a318d29a for earlier term 14. Current term is 16. Ops: [14.2146-14.2273]
I20260812 08:01:32.499303 11225 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 }, Status: INVALID_TERM, Last received: 14.2272, Next index: 2273, Last known committed idx: 2265, Time since last communication: 0.000s
I20260812 08:01:32.499538 11221 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 }, Status: INVALID_TERM, Last received: 14.2188, Next index: 2189, Last known committed idx: 2180, Time since last communication: 0.000s
I20260812 08:01:32.500154 11221 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 }, Status: INVALID_TERM, Last received: 14.2145, Next index: 2146, Last known committed idx: 2144, Time since last communication: 0.000s
I20260812 08:01:32.500229 11192 raft_consensus.cc:3055] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 14 LEADER]: Stepping down as leader of term 14
I20260812 08:01:32.500306 11192 raft_consensus.cc:740] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 56b60424ed9f4a079689ad82a318d29a, State: Running, Role: LEADER
I20260812 08:01:32.500608 11192 consensus_queue.cc:260] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2272, Committed index: 2272, Last appended: 14.2273, Last appended by leader: 2273, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
W20260812 08:01:32.500677 10663 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58954: Illegal state: replica 56b60424ed9f4a079689ad82a318d29a is not leader of this config: current role FOLLOWER
I20260812 08:01:32.501041 11192 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 14 FOLLOWER]: Advancing to term 16
W20260812 08:01:32.501272 10662 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58954: Illegal state: replica 56b60424ed9f4a079689ad82a318d29a is not leader of this config: current role FOLLOWER
W20260812 08:01:32.502084 10669 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58954: Illegal state: replica 56b60424ed9f4a079689ad82a318d29a is not leader of this config: current role FOLLOWER
W20260812 08:01:32.502223 11240 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58954: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.013s ago.
W20260812 08:01:32.501076 11226 consensus_queue.cc:1175] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "faafbc72cbd444ff90977a2cdb091b72" responder_term: 16 status { last_received { term: 14 index: 2273 } last_committed_idx: 2265 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 56b60424ed9f4a079689ad82a318d29a for earlier term 14. Current term is 16. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260812 08:01:32.502579 10662 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58954: Illegal state: replica 56b60424ed9f4a079689ad82a318d29a is not leader of this config: current role FOLLOWER
I20260812 08:01:32.499449 11235 rpcz_store.cc:276] 0812 08:01:28.752210 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:28.752241 (+    31us) service_pool.cc:224] Handling call
0812 08:01:32.493140 (+3740899us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:28.752619 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:28.752713 (+    94us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:28.752718 (+     5us) write_op.cc:435] Acquired schema lock
0812 08:01:28.752719 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:28.752803 (+    84us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:28.752808 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:28.752811 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:28.752873 (+    62us) tablet.cc:701] Row locks acquired
0812 08:01:28.752874 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:28.752936 (+    62us) write_op.cc:270] Start()
0812 08:01:28.752957 (+    21us) write_op.cc:276] Timestamp: P: 1786521688752897 usec, L: 0
0812 08:01:28.752958 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:01:28.753090 (+   132us) log.cc:844] Serialized 696 byte log entry
0812 08:01:32.489849 (+3736759us) op_driver.cc:464] REPLICATION: finished
0812 08:01:32.492681 (+  2832us) write_op.cc:301] APPLY: starting
0812 08:01:32.492694 (+    13us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:32.492724 (+    30us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:32.492737 (+    13us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:32.492930 (+   193us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:32.492957 (+    27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:32.492962 (+     5us) write_op.cc:312] APPLY: finished
0812 08:01:32.493028 (+    66us) log.cc:844] Serialized 77 byte log entry
0812 08:01:32.493067 (+    39us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:32.493081 (+    14us) write_op.cc:454] Released schema lock
0812 08:01:32.493085 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2787,"num_ops":10,"prepare.queue_time_us":115,"prepare.run_cpu_time_us":509,"prepare.run_wall_time_us":516,"replication_time_us":3736876,"spinlock_wait_cycles":41088,"thread_start_us":64,"threads_started":1}]]}
W20260812 08:01:32.503000 10663 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58954: Illegal state: replica 56b60424ed9f4a079689ad82a318d29a is not leader of this config: current role FOLLOWER
W20260812 08:01:32.504478 10663 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58954: Illegal state: replica 56b60424ed9f4a079689ad82a318d29a is not leader of this config: current role FOLLOWER
I20260812 08:01:32.506635  9808 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 16 FOLLOWER]: Refusing update from remote peer 3ae3f7a6e9f943e996a6dcea504878d0: Log matching property violated. Preceding OpId in replica: term: 14 index: 2272. Preceding OpId from leader: term: 16 index: 2275. (index mismatch)
I20260812 08:01:32.506786 10079 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 16 FOLLOWER]: Refusing update from remote peer 3ae3f7a6e9f943e996a6dcea504878d0: Log matching property violated. Preceding OpId in replica: term: 14 index: 2273. Preceding OpId from leader: term: 16 index: 2275. (index mismatch)
I20260812 08:01:32.506819  8004 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 16 FOLLOWER]: Refusing update from remote peer 3ae3f7a6e9f943e996a6dcea504878d0: Log matching property violated. Preceding OpId in replica: term: 14 index: 2145. Preceding OpId from leader: term: 16 index: 2275. (index mismatch)
I20260812 08:01:32.507128 10727 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 16 FOLLOWER]: Refusing update from remote peer 3ae3f7a6e9f943e996a6dcea504878d0: Log matching property violated. Preceding OpId in replica: term: 14 index: 2273. Preceding OpId from leader: term: 16 index: 2275. (index mismatch)
I20260812 08:01:32.507273 11209 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2274, Last known committed idx: 2144, Time since last communication: 0.000s
I20260812 08:01:32.507431 11210 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2274, Last known committed idx: 2265, Time since last communication: 0.000s
I20260812 08:01:32.507568  8220 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 16 FOLLOWER]: Refusing update from remote peer 3ae3f7a6e9f943e996a6dcea504878d0: Log matching property violated. Preceding OpId in replica: term: 14 index: 2226. Preceding OpId from leader: term: 16 index: 2275. (index mismatch)
I20260812 08:01:32.508100 11242 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2274, Last known committed idx: 2272, Time since last communication: 0.000s
I20260812 08:01:32.508253 11243 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2274, Last known committed idx: 2265, Time since last communication: 0.000s
I20260812 08:01:32.508562  7381 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 16 FOLLOWER]: Refusing update from remote peer 3ae3f7a6e9f943e996a6dcea504878d0: Log matching property violated. Preceding OpId in replica: term: 14 index: 2188. Preceding OpId from leader: term: 16 index: 2275. (index mismatch)
I20260812 08:01:32.509008 11243 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2274, Last known committed idx: 2180, Time since last communication: 0.000s
I20260812 08:01:32.517987 11244 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2274, Last known committed idx: 2219, Time since last communication: 0.000s
I20260812 08:01:32.529862 11235 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:58954 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2261, attempt_no=0}) took 3772 ms. Trace:
I20260812 08:01:32.529958 11235 rpcz_store.cc:276] 0812 08:01:28.757674 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:28.757853 (+   179us) service_pool.cc:224] Handling call
0812 08:01:32.529843 (+3771990us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:28.758185 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:28.758264 (+    79us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:28.758267 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:28.758269 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:28.758350 (+    81us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:28.758356 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:28.758359 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:28.758426 (+    67us) tablet.cc:701] Row locks acquired
0812 08:01:28.758427 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:28.758455 (+    28us) write_op.cc:270] Start()
0812 08:01:28.758473 (+    18us) write_op.cc:276] Timestamp: P: 1786521688758417 usec, L: 0
0812 08:01:28.758474 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:01:28.758586 (+   112us) log.cc:844] Serialized 696 byte log entry
0812 08:01:32.528000 (+3769414us) op_driver.cc:464] REPLICATION: finished
0812 08:01:32.529302 (+  1302us) write_op.cc:301] APPLY: starting
0812 08:01:32.529317 (+    15us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:32.529358 (+    41us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:32.529369 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:32.529587 (+   218us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:32.529621 (+    34us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:32.529626 (+     5us) write_op.cc:312] APPLY: finished
0812 08:01:32.529693 (+    67us) log.cc:844] Serialized 77 byte log entry
0812 08:01:32.529734 (+    41us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:32.529751 (+    17us) write_op.cc:454] Released schema lock
0812 08:01:32.529756 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1230,"num_ops":10,"prepare.queue_time_us":45,"prepare.run_cpu_time_us":451,"prepare.run_wall_time_us":451,"replication_time_us":3769502}]]}
I20260812 08:01:34.404587 11323 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3ae3f7a6e9f943e996a6dcea504878d0)
I20260812 08:01:34.404732 11323 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:34.405694 11323 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183)
I20260812 08:01:34.406080  9809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 17 candidate_status { last_received { term: 16 index: 2347 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:34.406071 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 17 candidate_status { last_received { term: 16 index: 2347 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:34.406138  8220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 17 candidate_status { last_received { term: 16 index: 2347 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:34.406385 10728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 17 candidate_status { last_received { term: 16 index: 2347 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:34.406533  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 17 candidate_status { last_received { term: 16 index: 2347 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:34.406827  7902 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:34.407008 11323 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260812 08:01:34.715662 11324 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3ae3f7a6e9f943e996a6dcea504878d0)
I20260812 08:01:34.715842 11324 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:34.716584 11324 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:34.717069  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 17 candidate_status { last_received { term: 16 index: 2455 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:34.717110 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 17 candidate_status { last_received { term: 16 index: 2455 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:34.717196  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 17 candidate_status { last_received { term: 16 index: 2455 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:34.717355  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 16.
I20260812 08:01:34.717448  9809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 17 candidate_status { last_received { term: 16 index: 2455 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:34.717474  8220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 17 candidate_status { last_received { term: 16 index: 2455 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:34.717906 10629 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 56b60424ed9f4a079689ad82a318d29a, 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:34.718091 11324 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260812 08:01:34.728374 11325 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3ae3f7a6e9f943e996a6dcea504878d0)
I20260812 08:01:34.728533 11325 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:34.729197 11325 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:34.729664  8220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 17 candidate_status { last_received { term: 16 index: 2460 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:34.729686  9809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 17 candidate_status { last_received { term: 16 index: 2460 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:34.729720  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 17 candidate_status { last_received { term: 16 index: 2460 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:34.729842  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 16.
I20260812 08:01:34.729847  9809 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 16.
I20260812 08:01:34.729915  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 17 candidate_status { last_received { term: 16 index: 2460 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:34.730087 11326 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3ae3f7a6e9f943e996a6dcea504878d0)
I20260812 08:01:34.730041 10728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 17 candidate_status { last_received { term: 16 index: 2460 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:34.730201 10728 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 16.
I20260812 08:01:34.730178 11326 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:34.730544  9966 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 56b60424ed9f4a079689ad82a318d29a, 780d7e64a8214418a371f35351b113df, faafbc72cbd444ff90977a2cdb091b72; no voters: 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6
I20260812 08:01:34.730706 11325 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260812 08:01:34.730759 11325 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3ae3f7a6e9f943e996a6dcea504878d0)
I20260812 08:01:34.730786 11325 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 16 FOLLOWER]: Advancing to term 17
I20260812 08:01:34.731107 11326 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:34.731477 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 17 candidate_status { last_received { term: 16 index: 2460 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:34.731490 10728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 17 candidate_status { last_received { term: 16 index: 2460 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:34.731593 10728 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 16.
I20260812 08:01:34.731599  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 17 candidate_status { last_received { term: 16 index: 2460 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:34.731714  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 16.
I20260812 08:01:34.731746  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 17 candidate_status { last_received { term: 16 index: 2460 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:34.731817  8220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 17 candidate_status { last_received { term: 16 index: 2460 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:34.732386 11325 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:34.732616 10079 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1664e32d3cd0473592085aa522379f22 in current term 17: Already voted for candidate faafbc72cbd444ff90977a2cdb091b72 in this term.
I20260812 08:01:34.733181 11325 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 17 election: Requested vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:34.733312  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 17 candidate_status { last_received { term: 16 index: 2460 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:01:34.733330  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 17 candidate_status { last_received { term: 16 index: 2460 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:01:34.733423  8004 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 16 FOLLOWER]: Advancing to term 17
I20260812 08:01:34.733430  8220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 17 candidate_status { last_received { term: 16 index: 2460 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
I20260812 08:01:34.733508 10728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 17 candidate_status { last_received { term: 16 index: 2460 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a"
I20260812 08:01:34.733593 10728 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 16 FOLLOWER]: Advancing to term 17
I20260812 08:01:34.733733  9809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 17 candidate_status { last_received { term: 16 index: 2460 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22"
I20260812 08:01:34.733845  9809 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 16 FOLLOWER]: Advancing to term 17
I20260812 08:01:34.734361  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 17.
I20260812 08:01:34.734722 10728 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 17.
I20260812 08:01:34.734781  9809 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 17.
I20260812 08:01:34.735042  9966 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 56b60424ed9f4a079689ad82a318d29a, 780d7e64a8214418a371f35351b113df, faafbc72cbd444ff90977a2cdb091b72; no voters: 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6
I20260812 08:01:34.735227 11325 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 17 FOLLOWER]: Leader election won for term 17
I20260812 08:01:34.735399 11325 raft_consensus.cc:697] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 17 LEADER]: Becoming Leader. State: Replica: faafbc72cbd444ff90977a2cdb091b72, State: Running, Role: LEADER
I20260812 08:01:34.735771 11325 consensus_queue.cc:237] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2452, Committed index: 2452, Last appended: 16.2460, Last appended by leader: 2460, Current term: 17, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:34.738358  6748 catalog_manager.cc:5719] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 reported cstate change: term changed from 16 to 17, leader changed from 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129) to faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132). New cstate: current_term: 17 leader_uuid: "faafbc72cbd444ff90977a2cdb091b72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } health_report { overall_health: UNKNOWN } } }
W20260812 08:01:34.805315 10363 connection.cc:256] RPC call timeout handler was delayed by -1.0023594389203936s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 08:01:34.805974 10460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 17 candidate_status { last_received { term: 16 index: 2460 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:34.806051 10462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 17 candidate_status { last_received { term: 16 index: 2460 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0"
I20260812 08:01:34.806414 10465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 17 candidate_status { last_received { term: 16 index: 2347 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:34.806703 10476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 17 candidate_status { last_received { term: 16 index: 2460 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:34.807235 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 17 candidate_status { last_received { term: 16 index: 2455 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:34.808012  9711 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 56b60424ed9f4a079689ad82a318d29a, 780d7e64a8214418a371f35351b113df; no voters: 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:34.808830 11326 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260812 08:01:34.809033  9809 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 17 FOLLOWER]: Rejecting Update request from peer 3ae3f7a6e9f943e996a6dcea504878d0 for earlier term 16. Current term is 17. Ops: []
I20260812 08:01:34.809995 10728 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 17 FOLLOWER]: Rejecting Update request from peer 3ae3f7a6e9f943e996a6dcea504878d0 for earlier term 16. Current term is 17. Ops: [16.2456-16.2460]
I20260812 08:01:34.811390 11341 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2441, attempt_no=0}) took 1600 ms. Trace:
I20260812 08:01:34.811498 11341 rpcz_store.cc:276] 0812 08:01:33.210712 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:33.210853 (+   141us) service_pool.cc:224] Handling call
0812 08:01:34.811375 (+1600522us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:33.211890 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:33.211971 (+    81us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:33.211975 (+     4us) write_op.cc:435] Acquired schema lock
0812 08:01:33.211977 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:33.212067 (+    90us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:33.212073 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:33.212077 (+     4us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:33.212143 (+    66us) tablet.cc:701] Row locks acquired
0812 08:01:33.212144 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:33.212177 (+    33us) write_op.cc:270] Start()
0812 08:01:33.212198 (+    21us) write_op.cc:276] Timestamp: P: 1786521693212097 usec, L: 0
0812 08:01:33.212199 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:01:33.212317 (+   118us) log.cc:844] Serialized 686 byte log entry
0812 08:01:34.808548 (+1596231us) op_driver.cc:464] REPLICATION: finished
0812 08:01:34.810864 (+  2316us) write_op.cc:301] APPLY: starting
0812 08:01:34.810878 (+    14us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:34.810911 (+    33us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:34.810924 (+    13us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:34.811141 (+   217us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:34.811169 (+    28us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:34.811174 (+     5us) write_op.cc:312] APPLY: finished
0812 08:01:34.811234 (+    60us) log.cc:844] Serialized 77 byte log entry
0812 08:01:34.811275 (+    41us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:34.811295 (+    20us) write_op.cc:454] Released schema lock
0812 08:01:34.811303 (+     8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2268,"num_ops":10,"prepare.queue_time_us":742,"prepare.run_cpu_time_us":519,"prepare.run_wall_time_us":518,"raft.queue_time_us":655,"raft.run_cpu_time_us":321,"raft.run_wall_time_us":727,"replication_time_us":1596331,"spinlock_wait_cycles":768,"thread_start_us":51,"threads_started":1}]]}
I20260812 08:01:34.811416 11342 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2442, attempt_no=0}) took 1600 ms. Trace:
I20260812 08:01:34.812053 11342 rpcz_store.cc:276] 0812 08:01:33.210939 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:33.210984 (+    45us) service_pool.cc:224] Handling call
0812 08:01:34.811404 (+1600420us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:33.212430 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:33.212518 (+    88us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:33.212521 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:33.212523 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:33.212612 (+    89us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:33.212619 (+     7us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:33.212622 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:33.212683 (+    61us) tablet.cc:701] Row locks acquired
0812 08:01:33.212685 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:33.212716 (+    31us) write_op.cc:270] Start()
0812 08:01:33.212736 (+    20us) write_op.cc:276] Timestamp: P: 1786521693212636 usec, L: 0
0812 08:01:33.212738 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:33.212850 (+   112us) log.cc:844] Serialized 696 byte log entry
0812 08:01:34.808650 (+1595800us) op_driver.cc:464] REPLICATION: finished
0812 08:01:34.810933 (+  2283us) write_op.cc:301] APPLY: starting
0812 08:01:34.810945 (+    12us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:34.810975 (+    30us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:34.810987 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:34.811166 (+   179us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:34.811193 (+    27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:34.811197 (+     4us) write_op.cc:312] APPLY: finished
0812 08:01:34.811250 (+    53us) log.cc:844] Serialized 77 byte log entry
0812 08:01:34.811290 (+    40us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:34.811304 (+    14us) write_op.cc:454] Released schema lock
0812 08:01:34.811310 (+     6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2241,"num_ops":10,"prepare.queue_time_us":1239,"prepare.run_cpu_time_us":516,"prepare.run_wall_time_us":516,"replication_time_us":1595896,"spinlock_wait_cycles":124416,"thread_start_us":47,"threads_started":1}]]}
I20260812 08:01:34.812974  8004 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 17 FOLLOWER]: Rejecting Update request from peer 3ae3f7a6e9f943e996a6dcea504878d0 for earlier term 16. Current term is 17. Ops: [16.2348-16.2460]
I20260812 08:01:34.814635 11270 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 }, Status: INVALID_TERM, Last received: 16.2455, Next index: 2456, Last known committed idx: 2452, Time since last communication: 0.000s
I20260812 08:01:34.814826 11270 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 }, Status: INVALID_TERM, Last received: 16.2460, Next index: 2461, Last known committed idx: 2452, Time since last communication: 0.000s
I20260812 08:01:34.814947 11270 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 }, Status: INVALID_TERM, Last received: 16.2347, Next index: 2348, Last known committed idx: 2346, Time since last communication: 0.000s
I20260812 08:01:34.815150 11270 raft_consensus.cc:3055] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 16 LEADER]: Stepping down as leader of term 16
I20260812 08:01:34.815199 11270 raft_consensus.cc:740] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3ae3f7a6e9f943e996a6dcea504878d0, State: Running, Role: LEADER
I20260812 08:01:34.815368 11270 consensus_queue.cc:260] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2455, Committed index: 2455, Last appended: 16.2460, Last appended by leader: 2460, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:34.815635 11270 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 16 FOLLOWER]: Advancing to term 17
I20260812 08:01:34.817200 11344 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2440, attempt_no=0}) took 1613 ms. Trace:
I20260812 08:01:34.817265 11344 rpcz_store.cc:276] 0812 08:01:33.203496 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:33.203540 (+    44us) service_pool.cc:224] Handling call
0812 08:01:34.817187 (+1613647us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:33.205618 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:33.205700 (+    82us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:33.205704 (+     4us) write_op.cc:435] Acquired schema lock
0812 08:01:33.205706 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:33.205797 (+    91us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:33.205803 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:33.205807 (+     4us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:33.205876 (+    69us) tablet.cc:701] Row locks acquired
0812 08:01:33.205878 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:33.205910 (+    32us) write_op.cc:270] Start()
0812 08:01:33.205929 (+    19us) write_op.cc:276] Timestamp: P: 1786521693205831 usec, L: 0
0812 08:01:33.205931 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:33.206048 (+   117us) log.cc:844] Serialized 696 byte log entry
0812 08:01:34.808362 (+1602314us) op_driver.cc:464] REPLICATION: finished
0812 08:01:34.809955 (+  1593us) write_op.cc:301] APPLY: starting
0812 08:01:34.809975 (+    20us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:34.810020 (+    45us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:34.810033 (+    13us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:34.810229 (+   196us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:34.810262 (+    33us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:34.810267 (+     5us) write_op.cc:312] APPLY: finished
0812 08:01:34.810342 (+    75us) log.cc:844] Serialized 77 byte log entry
0812 08:01:34.816785 (+  6443us) spinlock_profiling.cc:243] Waited 5.95 ms on lock 0x563343b12b9c. stack: 00007fe58dfd0d35 00007fe58dfd1325 00007fe58dadd82d 00005633135a4b18 00005633135a4b6b 00005633135a89f2 00007fe58df48a67 00007fe58df4965f 00007fe58fddace5 00007fe58fddad52 00007fe58dfe3cc6 000056331362f136 00007fe58dff4d84 00007fe58dff35a2 00007fe58dff2dfd 00007fe58fdc9ec4
0812 08:01:34.816887 (+   102us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:34.817113 (+   226us) write_op.cc:454] Released schema lock
0812 08:01:34.817120 (+     7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1538,"num_ops":10,"prepare.queue_time_us":1796,"prepare.run_cpu_time_us":515,"prepare.run_wall_time_us":515,"raft.queue_time_us":68,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":1500,"replication_time_us":1602399,"spinlock_wait_cycles":13677824,"thread_start_us":6404,"threads_started":2}]]}
W20260812 08:01:34.817612 10416 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:34.818171 10412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
I20260812 08:01:34.818488 10079 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 17 LEADER]: Rejecting Update request from peer 3ae3f7a6e9f943e996a6dcea504878d0 for earlier term 16. Current term is 17. Ops: []
W20260812 08:01:34.820122 10412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
I20260812 08:01:34.828529 10462 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 17 FOLLOWER]: Refusing update from remote peer faafbc72cbd444ff90977a2cdb091b72: Log matching property violated. Preceding OpId in replica: term: 16 index: 2460. Preceding OpId from leader: term: 17 index: 2462. (index mismatch)
I20260812 08:01:34.828807 10728 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 17 FOLLOWER]: Refusing update from remote peer faafbc72cbd444ff90977a2cdb091b72: Log matching property violated. Preceding OpId in replica: term: 16 index: 2455. Preceding OpId from leader: term: 17 index: 2462. (index mismatch)
I20260812 08:01:34.829312  8004 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 17 FOLLOWER]: Refusing update from remote peer faafbc72cbd444ff90977a2cdb091b72: Log matching property violated. Preceding OpId in replica: term: 16 index: 2347. Preceding OpId from leader: term: 17 index: 2462. (index mismatch)
I20260812 08:01:34.829385  9809 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 17 FOLLOWER]: Refusing update from remote peer faafbc72cbd444ff90977a2cdb091b72: Log matching property violated. Preceding OpId in replica: term: 16 index: 2460. Preceding OpId from leader: term: 17 index: 2462. (index mismatch)
I20260812 08:01:34.829610 11325 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2461, Last known committed idx: 2455, Time since last communication: 0.000s
I20260812 08:01:34.829694  8220 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 16 FOLLOWER]: Advancing to term 17
I20260812 08:01:34.830035 11325 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2461, Last known committed idx: 2452, Time since last communication: 0.000s
I20260812 08:01:34.830371 11325 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2461, Last known committed idx: 2452, Time since last communication: 0.000s
I20260812 08:01:34.830909  8220 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 17 FOLLOWER]: Refusing update from remote peer faafbc72cbd444ff90977a2cdb091b72: Log matching property violated. Preceding OpId in replica: term: 16 index: 2460. Preceding OpId from leader: term: 17 index: 2462. (index mismatch)
I20260812 08:01:34.831817 11328 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2461, Last known committed idx: 2455, Time since last communication: 0.000s
I20260812 08:01:34.832137 11328 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2461, Last known committed idx: 2346, Time since last communication: 0.000s
I20260812 08:01:34.843248 11344 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2443, attempt_no=0}) took 1625 ms. Trace:
I20260812 08:01:34.843358 11344 rpcz_store.cc:276] 0812 08:01:33.217617 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:33.217658 (+    41us) service_pool.cc:224] Handling call
0812 08:01:34.843229 (+1625571us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:33.222919 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:33.223005 (+    86us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:33.223008 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:33.223010 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:33.223090 (+    80us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:33.223095 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:33.223097 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:33.223143 (+    46us) tablet.cc:701] Row locks acquired
0812 08:01:33.223144 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:33.223168 (+    24us) write_op.cc:270] Start()
0812 08:01:33.223184 (+    16us) write_op.cc:276] Timestamp: P: 1786521693223153 usec, L: 0
0812 08:01:33.223185 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:01:33.223313 (+   128us) log.cc:844] Serialized 696 byte log entry
0812 08:01:34.842097 (+1618784us) op_driver.cc:464] REPLICATION: finished
0812 08:01:34.842779 (+   682us) write_op.cc:301] APPLY: starting
0812 08:01:34.842794 (+    15us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:34.842827 (+    33us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:34.842840 (+    13us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:34.843019 (+   179us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:34.843050 (+    31us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:34.843055 (+     5us) write_op.cc:312] APPLY: finished
0812 08:01:34.843116 (+    61us) log.cc:844] Serialized 77 byte log entry
0812 08:01:34.843154 (+    38us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:34.843168 (+    14us) write_op.cc:454] Released schema lock
0812 08:01:34.843173 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":641,"num_ops":10,"prepare.queue_time_us":4970,"prepare.run_cpu_time_us":469,"prepare.run_wall_time_us":468,"raft.queue_time_us":29,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":176,"replication_time_us":1618890}]]}
I20260812 08:01:34.844163 11344 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2444, attempt_no=0}) took 1623 ms. Trace:
I20260812 08:01:34.844233 11344 rpcz_store.cc:276] 0812 08:01:33.220576 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:33.220615 (+    39us) service_pool.cc:224] Handling call
0812 08:01:34.844152 (+1623537us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:33.223410 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:33.223483 (+    73us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:33.223486 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:33.223487 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:33.223581 (+    94us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:33.223586 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:33.223589 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:33.223649 (+    60us) tablet.cc:701] Row locks acquired
0812 08:01:33.223650 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:33.223681 (+    31us) write_op.cc:270] Start()
0812 08:01:33.223700 (+    19us) write_op.cc:276] Timestamp: P: 1786521693223664 usec, L: 0
0812 08:01:33.223701 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:01:33.223842 (+   141us) log.cc:844] Serialized 686 byte log entry
0812 08:01:34.842159 (+1618317us) op_driver.cc:464] REPLICATION: finished
0812 08:01:34.843764 (+  1605us) write_op.cc:301] APPLY: starting
0812 08:01:34.843773 (+     9us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:34.843801 (+    28us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:34.843812 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:34.843975 (+   163us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:34.844001 (+    26us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:34.844006 (+     5us) write_op.cc:312] APPLY: finished
0812 08:01:34.844057 (+    51us) log.cc:844] Serialized 77 byte log entry
0812 08:01:34.844089 (+    32us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:34.844100 (+    11us) write_op.cc:454] Released schema lock
0812 08:01:34.844104 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1580,"num_ops":10,"prepare.queue_time_us":2547,"prepare.run_cpu_time_us":480,"prepare.run_wall_time_us":479,"replication_time_us":1618446}]]}
I20260812 08:01:34.844987 11344 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2445, attempt_no=0}) took 1622 ms. Trace:
I20260812 08:01:34.845044 11344 rpcz_store.cc:276] 0812 08:01:33.222478 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:33.222517 (+    39us) service_pool.cc:224] Handling call
0812 08:01:34.844976 (+1622459us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:33.223919 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:33.223990 (+    71us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:33.223993 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:33.223995 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:33.224075 (+    80us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:33.224080 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:33.224083 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:33.224141 (+    58us) tablet.cc:701] Row locks acquired
0812 08:01:33.224142 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:33.224169 (+    27us) write_op.cc:270] Start()
0812 08:01:33.224186 (+    17us) write_op.cc:276] Timestamp: P: 1786521693224153 usec, L: 0
0812 08:01:33.224188 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:33.224311 (+   123us) log.cc:844] Serialized 696 byte log entry
0812 08:01:34.842213 (+1617902us) op_driver.cc:464] REPLICATION: finished
0812 08:01:34.844569 (+  2356us) write_op.cc:301] APPLY: starting
0812 08:01:34.844578 (+     9us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:34.844605 (+    27us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:34.844615 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:34.844776 (+   161us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:34.844801 (+    25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:34.844805 (+     4us) write_op.cc:312] APPLY: finished
0812 08:01:34.844854 (+    49us) log.cc:844] Serialized 77 byte log entry
0812 08:01:34.844887 (+    33us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:34.844922 (+    35us) write_op.cc:454] Released schema lock
0812 08:01:34.844927 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2331,"num_ops":10,"prepare.queue_time_us":1143,"prepare.run_cpu_time_us":437,"prepare.run_wall_time_us":437,"replication_time_us":1617996}]]}
I20260812 08:01:34.845775 11344 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2446, attempt_no=0}) took 1620 ms. Trace:
I20260812 08:01:34.847254 11344 rpcz_store.cc:276] 0812 08:01:33.225496 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:33.225536 (+    40us) service_pool.cc:224] Handling call
0812 08:01:34.845766 (+1620230us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:33.225840 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:33.225912 (+    72us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:33.225916 (+     4us) write_op.cc:435] Acquired schema lock
0812 08:01:33.225917 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:33.225998 (+    81us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:33.226003 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:33.226006 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:33.226068 (+    62us) tablet.cc:701] Row locks acquired
0812 08:01:33.226070 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:33.226120 (+    50us) write_op.cc:270] Start()
0812 08:01:33.226139 (+    19us) write_op.cc:276] Timestamp: P: 1786521693226103 usec, L: 0
0812 08:01:33.226141 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:33.226238 (+    97us) log.cc:844] Serialized 696 byte log entry
0812 08:01:34.842252 (+1616014us) op_driver.cc:464] REPLICATION: finished
0812 08:01:34.845356 (+  3104us) write_op.cc:301] APPLY: starting
0812 08:01:34.845366 (+    10us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:34.845391 (+    25us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:34.845401 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:34.845591 (+   190us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:34.845617 (+    26us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:34.845621 (+     4us) write_op.cc:312] APPLY: finished
0812 08:01:34.845673 (+    52us) log.cc:844] Serialized 77 byte log entry
0812 08:01:34.845704 (+    31us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:34.845715 (+    11us) write_op.cc:454] Released schema lock
0812 08:01:34.845720 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3079,"num_ops":10,"prepare.queue_time_us":56,"prepare.run_cpu_time_us":445,"prepare.run_wall_time_us":445,"replication_time_us":1616100,"spinlock_wait_cycles":35584,"thread_start_us":105,"threads_started":1}]]}
I20260812 08:01:34.853066 11344 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2447, attempt_no=0}) took 1627 ms. Trace:
I20260812 08:01:34.853139 11344 rpcz_store.cc:276] 0812 08:01:33.225984 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:33.226021 (+    37us) service_pool.cc:224] Handling call
0812 08:01:34.853054 (+1627033us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:33.226307 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:33.226373 (+    66us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:33.226376 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:33.226377 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:33.226457 (+    80us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:33.226462 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:33.226465 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:33.226522 (+    57us) tablet.cc:701] Row locks acquired
0812 08:01:33.226535 (+    13us) write_op.cc:260] PREPARE: finished
0812 08:01:33.226569 (+    34us) write_op.cc:270] Start()
0812 08:01:33.226587 (+    18us) write_op.cc:276] Timestamp: P: 1786521693226552 usec, L: 0
0812 08:01:33.226588 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:01:33.226686 (+    98us) log.cc:844] Serialized 696 byte log entry
0812 08:01:34.842413 (+1615727us) op_driver.cc:464] REPLICATION: finished
0812 08:01:34.847608 (+  5195us) write_op.cc:301] APPLY: starting
0812 08:01:34.847618 (+    10us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:34.847643 (+    25us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:34.847654 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:34.847817 (+   163us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:34.847843 (+    26us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:34.847847 (+     4us) write_op.cc:312] APPLY: finished
0812 08:01:34.847900 (+    53us) log.cc:844] Serialized 77 byte log entry
0812 08:01:34.852973 (+  5073us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:34.852988 (+    15us) write_op.cc:454] Released schema lock
0812 08:01:34.852994 (+     6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":5167,"num_ops":10,"prepare.queue_time_us":57,"prepare.run_cpu_time_us":426,"prepare.run_wall_time_us":427,"replication_time_us":1615807}]]}
I20260812 08:01:34.889262  7378 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 16 FOLLOWER]: Advancing to term 17
I20260812 08:01:34.890560  7378 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 17 FOLLOWER]: Refusing update from remote peer faafbc72cbd444ff90977a2cdb091b72: Log matching property violated. Preceding OpId in replica: term: 16 index: 2460. Preceding OpId from leader: term: 17 index: 2462. (index mismatch)
I20260812 08:01:34.891604 11358 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2461, Last known committed idx: 2452, Time since last communication: 0.000s
I20260812 08:01:36.885577 11433 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:01:36.885742 11433 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:36.887068 10726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 18 candidate_status { last_received { term: 17 index: 2582 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:36.887040  9808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 18 candidate_status { last_received { term: 17 index: 2582 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:36.887043 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 18 candidate_status { last_received { term: 17 index: 2582 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:36.887065  8224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 18 candidate_status { last_received { term: 17 index: 2582 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:36.887589  7286 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 16901c5559a748a9aa770d73e254099c; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6
I20260812 08:01:36.887591 11433 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:36.887851 11433 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260812 08:01:36.888021  8006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 18 candidate_status { last_received { term: 17 index: 2582 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:37.128199 11435 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:01:37.128372 11435 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:37.129390 11436 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:01:37.129604 10726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 18 candidate_status { last_received { term: 17 index: 2674 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:37.129637 11435 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:37.129781  8224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 18 candidate_status { last_received { term: 17 index: 2674 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:37.129959 11436 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:37.130125  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 18 candidate_status { last_received { term: 17 index: 2674 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:37.130125  8006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 18 candidate_status { last_received { term: 17 index: 2674 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:37.130270  7378 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 17.
I20260812 08:01:37.129572 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 18 candidate_status { last_received { term: 17 index: 2674 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:37.130432 10472 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 17.
I20260812 08:01:37.130772 11436 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:37.131171  8224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 18 candidate_status { last_received { term: 17 index: 2674 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:37.131196 10726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 18 candidate_status { last_received { term: 17 index: 2674 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:37.131191  9808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 18 candidate_status { last_received { term: 17 index: 2674 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:37.131315  9808 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 17.
I20260812 08:01:37.131313  8006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 18 candidate_status { last_received { term: 17 index: 2674 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:37.131601  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 18 candidate_status { last_received { term: 17 index: 2674 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:37.131722  7378 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 17.
I20260812 08:01:37.462705 11437 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 17 FOLLOWER]: Starting pre-election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:01:37.462874 11437 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:37.463598 11437 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183)
I20260812 08:01:37.464072 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 18 candidate_status { last_received { term: 17 index: 2551 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:37.464038  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 18 candidate_status { last_received { term: 17 index: 2551 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:37.464087 10726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 18 candidate_status { last_received { term: 17 index: 2551 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:37.464087  8224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 18 candidate_status { last_received { term: 17 index: 2551 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:37.464250  7378 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df for term 18 because replica has last-logged OpId of term: 17 index: 2582, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 2551.
I20260812 08:01:37.464252 10472 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df for term 18 because replica has last-logged OpId of term: 17 index: 2674, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 2551.
I20260812 08:01:37.464740  7905 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 780d7e64a8214418a371f35351b113df; no voters: 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6
I20260812 08:01:37.464959 11437 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260812 08:01:37.464948  9808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 18 candidate_status { last_received { term: 17 index: 2551 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:37.465118  9808 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df for term 18 because replica has last-logged OpId of term: 17 index: 2674, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 2551.
I20260812 08:01:37.466857 11439 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:01:37.466974 11439 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:37.467330 11440 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:01:37.467430 11440 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:37.467890 11439 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:37.468279 10462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 18 candidate_status { last_received { term: 17 index: 2668 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:37.468454 10462 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56b60424ed9f4a079689ad82a318d29a for term 18 because replica has last-logged OpId of term: 17 index: 2674, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 2668.
I20260812 08:01:37.468449 10726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 2625 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:37.468523  8006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 2625 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:37.468614 10726 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6801d45ca6614156906e5d571ebcb8a6 for term 18 because replica has last-logged OpId of term: 17 index: 2668, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 2625.
I20260812 08:01:37.468636  8006 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6801d45ca6614156906e5d571ebcb8a6 in term 17.
I20260812 08:01:37.468672 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 2625 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:37.468683  9808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 2625 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:37.468803 10472 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6801d45ca6614156906e5d571ebcb8a6 for term 18 because replica has last-logged OpId of term: 17 index: 2674, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 2625.
I20260812 08:01:37.468818  9808 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6801d45ca6614156906e5d571ebcb8a6 for term 18 because replica has last-logged OpId of term: 17 index: 2674, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 2625.
I20260812 08:01:37.469061  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 2625 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:37.469215  7378 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6801d45ca6614156906e5d571ebcb8a6 in term 17.
I20260812 08:01:37.469450  9808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 18 candidate_status { last_received { term: 17 index: 2668 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:37.469586  9808 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56b60424ed9f4a079689ad82a318d29a for term 18 because replica has last-logged OpId of term: 17 index: 2674, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 2668.
I20260812 08:01:37.469619 11440 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:37.469753  8224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 18 candidate_status { last_received { term: 17 index: 2668 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:37.469789  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 18 candidate_status { last_received { term: 17 index: 2668 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:37.469866  8224 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 17.
I20260812 08:01:37.469920  7378 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 17.
I20260812 08:01:37.469930  8006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 18 candidate_status { last_received { term: 17 index: 2668 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:37.470029  8006 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 17.
I20260812 08:01:37.470330 10631 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0
I20260812 08:01:37.470530 11440 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260812 08:01:37.470603 11440 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 17 FOLLOWER]: Starting leader election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:01:37.470633 11440 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 17 FOLLOWER]: Advancing to term 18
I20260812 08:01:37.471513 11440 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:37.472220 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 18 candidate_status { last_received { term: 17 index: 2668 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0"
I20260812 08:01:37.472280  9808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 18 candidate_status { last_received { term: 17 index: 2668 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22"
I20260812 08:01:37.472333 10472 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 17 FOLLOWER]: Advancing to term 18
I20260812 08:01:37.472316  8224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 18 candidate_status { last_received { term: 17 index: 2668 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
I20260812 08:01:37.472361  9808 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 17 FOLLOWER]: Advancing to term 18
I20260812 08:01:37.472424  8224 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 17 FOLLOWER]: Advancing to term 18
I20260812 08:01:37.472633 11440 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 18 election: Requested vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:37.473416 10472 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 56b60424ed9f4a079689ad82a318d29a for term 18 because replica has last-logged OpId of term: 17 index: 2674, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 2668.
I20260812 08:01:37.473415  9808 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 56b60424ed9f4a079689ad82a318d29a for term 18 because replica has last-logged OpId of term: 17 index: 2674, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 2668.
I20260812 08:01:37.473461  8224 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 18.
I20260812 08:01:37.473897  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 18 candidate_status { last_received { term: 17 index: 2668 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:01:37.473899  8006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 18 candidate_status { last_received { term: 17 index: 2668 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:01:37.474011  8006 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 17 FOLLOWER]: Advancing to term 18
I20260812 08:01:37.474005  7378 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 17 FOLLOWER]: Advancing to term 18
I20260812 08:01:37.474771  7378 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 18.
I20260812 08:01:37.474777  8006 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 18.
I20260812 08:01:37.475055 10631 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0
I20260812 08:01:37.475241 11440 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 18 FOLLOWER]: Leader election won for term 18
I20260812 08:01:37.475421 11440 raft_consensus.cc:697] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 18 LEADER]: Becoming Leader. State: Replica: 56b60424ed9f4a079689ad82a318d29a, State: Running, Role: LEADER
I20260812 08:01:37.475582 11440 consensus_queue.cc:237] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2663, Committed index: 2663, Last appended: 17.2668, Last appended by leader: 2668, Current term: 18, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:37.477563  6748 catalog_manager.cc:5719] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a reported cstate change: term changed from 17 to 18, leader changed from faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132) to 56b60424ed9f4a079689ad82a318d29a (127.5.142.130). New cstate: current_term: 18 leader_uuid: "56b60424ed9f4a079689ad82a318d29a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } health_report { overall_health: UNKNOWN } } }
W20260812 08:01:37.807298  9965 connection.cc:256] RPC call timeout handler was delayed by -1.6193095520138741s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 08:01:37.807816  9966 connection.cc:256] RPC call timeout handler was delayed by -1.2879564750473946s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 08:01:37.808058  9968 connection.cc:256] RPC call timeout handler was delayed by -1.4005468140821904s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 08:01:37.808759  9969 connection.cc:256] RPC call timeout handler was delayed by -1.5271862992085516s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 08:01:37.810001 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 18 candidate_status { last_received { term: 17 index: 2674 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:37.810357 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 18 candidate_status { last_received { term: 17 index: 2674 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:37.810612 10060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 18 candidate_status { last_received { term: 17 index: 2551 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:37.810858 10063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 2625 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:37.811101 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 18 candidate_status { last_received { term: 17 index: 2668 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72"
I20260812 08:01:37.811291 10065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 18 candidate_status { last_received { term: 17 index: 2668 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:37.812695 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 18 candidate_status { last_received { term: 17 index: 2582 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:37.813558  8224 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 18 FOLLOWER]: Rejecting Update request from peer faafbc72cbd444ff90977a2cdb091b72 for earlier term 17. Current term is 18. Ops: [17.2626-17.2674]
I20260812 08:01:37.814436 11421 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 }, Status: INVALID_TERM, Last received: 17.2625, Next index: 2626, Last known committed idx: 2625, Time since last communication: 0.000s
I20260812 08:01:37.814595 11421 raft_consensus.cc:3055] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 17 LEADER]: Stepping down as leader of term 17
I20260812 08:01:37.814649 11421 raft_consensus.cc:740] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: faafbc72cbd444ff90977a2cdb091b72, State: Running, Role: LEADER
I20260812 08:01:37.814658 11376 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59446 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2653, attempt_no=0}) took 2201 ms. Trace:
I20260812 08:01:37.814697 11384 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59446 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2654, attempt_no=0}) took 2200 ms. Trace:
I20260812 08:01:37.814813 11421 consensus_queue.cc:260] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2668, Committed index: 2668, Last appended: 17.2674, Last appended by leader: 2674, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:37.815102 11421 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 17 FOLLOWER]: Advancing to term 18
I20260812 08:01:37.814736 11376 rpcz_store.cc:276] 0812 08:01:35.612770 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:35.613286 (+   516us) service_pool.cc:224] Handling call
0812 08:01:37.814629 (+2201343us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:35.613637 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:35.613713 (+    76us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:35.613716 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:35.613717 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:35.613790 (+    73us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:35.613794 (+     4us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:35.613796 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:35.613841 (+    45us) tablet.cc:701] Row locks acquired
0812 08:01:35.613842 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:35.613874 (+    32us) write_op.cc:270] Start()
0812 08:01:35.613895 (+    21us) write_op.cc:276] Timestamp: P: 1786521695613855 usec, L: 0
0812 08:01:35.613897 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:35.614016 (+   119us) log.cc:844] Serialized 696 byte log entry
0812 08:01:37.811819 (+2197803us) op_driver.cc:464] REPLICATION: finished
0812 08:01:37.813921 (+  2102us) write_op.cc:301] APPLY: starting
0812 08:01:37.813952 (+    31us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:37.813995 (+    43us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:37.814008 (+    13us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:37.814226 (+   218us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:37.814262 (+    36us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:37.814267 (+     5us) write_op.cc:312] APPLY: finished
0812 08:01:37.814350 (+    83us) log.cc:844] Serialized 77 byte log entry
0812 08:01:37.814531 (+   181us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:37.814549 (+    18us) write_op.cc:454] Released schema lock
0812 08:01:37.814556 (+     7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2039,"num_ops":10,"prepare.queue_time_us":33,"prepare.run_cpu_time_us":455,"prepare.run_wall_time_us":455,"raft.queue_time_us":29,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":154,"replication_time_us":2197889,"thread_start_us":104,"threads_started":1}]]}
I20260812 08:01:37.815928 10726 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 18 LEADER]: Rejecting Update request from peer faafbc72cbd444ff90977a2cdb091b72 for earlier term 17. Current term is 18. Ops: [17.2669-17.2674]
I20260812 08:01:37.816076  8006 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 18 FOLLOWER]: Rejecting Update request from peer faafbc72cbd444ff90977a2cdb091b72 for earlier term 17. Current term is 18. Ops: [17.2552-17.2674]
I20260812 08:01:37.816128  9808 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 18 FOLLOWER]: Rejecting Update request from peer faafbc72cbd444ff90977a2cdb091b72 for earlier term 17. Current term is 18. Ops: []
I20260812 08:01:37.814742 11384 rpcz_store.cc:276] 0812 08:01:35.614498 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:35.615351 (+   853us) service_pool.cc:224] Handling call
0812 08:01:37.814674 (+2199323us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:35.616136 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:35.616222 (+    86us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:35.616226 (+     4us) write_op.cc:435] Acquired schema lock
0812 08:01:35.616228 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:35.616317 (+    89us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:35.616323 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:35.616326 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:35.616387 (+    61us) tablet.cc:701] Row locks acquired
0812 08:01:35.616389 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:35.616420 (+    31us) write_op.cc:270] Start()
0812 08:01:35.616441 (+    21us) write_op.cc:276] Timestamp: P: 1786521695616401 usec, L: 0
0812 08:01:35.616443 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:35.616557 (+   114us) log.cc:844] Serialized 686 byte log entry
0812 08:01:37.811865 (+2195308us) op_driver.cc:464] REPLICATION: finished
0812 08:01:37.814261 (+  2396us) write_op.cc:301] APPLY: starting
0812 08:01:37.814274 (+    13us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:37.814302 (+    28us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:37.814314 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:37.814508 (+   194us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:37.814535 (+    27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:37.814539 (+     4us) write_op.cc:312] APPLY: finished
0812 08:01:37.814591 (+    52us) log.cc:844] Serialized 77 byte log entry
0812 08:01:37.814622 (+    31us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:37.814631 (+     9us) write_op.cc:454] Released schema lock
0812 08:01:37.814635 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2354,"num_ops":10,"prepare.queue_time_us":494,"prepare.run_cpu_time_us":520,"prepare.run_wall_time_us":1864,"raft.queue_time_us":74,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":168,"replication_time_us":2195413}]]}
I20260812 08:01:37.817067 10363 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0; no voters: 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:37.817231  9706 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0; no voters: 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:37.818063 11453 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260812 08:01:37.819154  8112 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:37.819491 11439 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260812 08:01:37.819656  7378 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 18 FOLLOWER]: Rejecting Update request from peer faafbc72cbd444ff90977a2cdb091b72 for earlier term 17. Current term is 18. Ops: [17.2583-17.2674]
W20260812 08:01:37.819738 10009 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Illegal state: replica faafbc72cbd444ff90977a2cdb091b72 is not leader of this config: current role FOLLOWER
W20260812 08:01:37.820351 10005 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Illegal state: replica faafbc72cbd444ff90977a2cdb091b72 is not leader of this config: current role FOLLOWER
I20260812 08:01:37.820508 10472 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 18 FOLLOWER]: Rejecting Update request from peer faafbc72cbd444ff90977a2cdb091b72 for earlier term 17. Current term is 18. Ops: []
I20260812 08:01:37.820868 11451 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260812 08:01:37.823897 10412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:37.825982 10412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:37.827160  9748 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48992: Illegal state: replica 1664e32d3cd0473592085aa522379f22 is not leader of this config: current role FOLLOWER
W20260812 08:01:37.833340  9748 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48992: Illegal state: replica 1664e32d3cd0473592085aa522379f22 is not leader of this config: current role FOLLOWER
I20260812 08:01:37.836026  7378 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 18 FOLLOWER]: Refusing update from remote peer 56b60424ed9f4a079689ad82a318d29a: Log matching property violated. Preceding OpId in replica: term: 17 index: 2582. Preceding OpId from leader: term: 18 index: 2670. (index mismatch)
I20260812 08:01:37.836355  8006 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 18 FOLLOWER]: Refusing update from remote peer 56b60424ed9f4a079689ad82a318d29a: Log matching property violated. Preceding OpId in replica: term: 17 index: 2551. Preceding OpId from leader: term: 18 index: 2670. (index mismatch)
I20260812 08:01:37.836484  9808 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 18 FOLLOWER]: Refusing update from remote peer 56b60424ed9f4a079689ad82a318d29a: Log matching property violated. Preceding OpId in replica: term: 17 index: 2674. Preceding OpId from leader: term: 18 index: 2670. (term mismatch)
I20260812 08:01:37.836545  9808 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting all ops after (but not including) 2669
I20260812 08:01:37.836520 10472 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 18 FOLLOWER]: Refusing update from remote peer 56b60424ed9f4a079689ad82a318d29a: Log matching property violated. Preceding OpId in replica: term: 17 index: 2674. Preceding OpId from leader: term: 18 index: 2670. (term mismatch)
I20260812 08:01:37.836529 10082 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 18 FOLLOWER]: Refusing update from remote peer 56b60424ed9f4a079689ad82a318d29a: Log matching property violated. Preceding OpId in replica: term: 17 index: 2674. Preceding OpId from leader: term: 18 index: 2670. (term mismatch)
I20260812 08:01:37.836568  9808 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting uncommitted WRITE_OP operation due to leader change: 17.2670
I20260812 08:01:37.836580 10472 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting all ops after (but not including) 2669
I20260812 08:01:37.836596 10082 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting all ops after (but not including) 2669
I20260812 08:01:37.836604 10472 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 17.2670
I20260812 08:01:37.836621 10082 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 17.2670
W20260812 08:01:37.836728 10082 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Aborted: Op aborted by new leader
I20260812 08:01:37.836753  9808 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting uncommitted WRITE_OP operation due to leader change: 17.2671
I20260812 08:01:37.836848 10472 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 17.2671
I20260812 08:01:37.836853 10082 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59446 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2656, attempt_no=0}) took 2218 ms. Trace:
I20260812 08:01:37.836925  9808 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting uncommitted WRITE_OP operation due to leader change: 17.2672
I20260812 08:01:37.837072  9808 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting uncommitted WRITE_OP operation due to leader change: 17.2673
I20260812 08:01:37.837074 10472 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 17.2672
I20260812 08:01:37.836936 10082 rpcz_store.cc:276] 0812 08:01:35.618666 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:35.618697 (+    31us) service_pool.cc:224] Handling call
0812 08:01:37.836843 (+2218146us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:35.619077 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:35.619151 (+    74us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:35.619154 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:35.619156 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:35.619235 (+    79us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:35.619241 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:35.619244 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:35.619301 (+    57us) tablet.cc:701] Row locks acquired
0812 08:01:35.619302 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:35.619331 (+    29us) write_op.cc:270] Start()
0812 08:01:35.619352 (+    21us) write_op.cc:276] Timestamp: P: 1786521695619313 usec, L: 0
0812 08:01:35.619353 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:01:35.619466 (+   113us) log.cc:844] Serialized 696 byte log entry
0812 08:01:37.836683 (+2217217us) op_driver.cc:464] REPLICATION: finished
0812 08:01:37.836689 (+     6us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 08:01:37.836693 (+     4us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:37.836717 (+    24us) write_op.cc:454] Released schema lock
0812 08:01:37.836723 (+     6us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":162,"prepare.run_cpu_time_us":438,"prepare.run_wall_time_us":437,"replication_time_us":2217292}]]}
I20260812 08:01:37.837206  9808 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting uncommitted WRITE_OP operation due to leader change: 17.2674
I20260812 08:01:37.837218 10082 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 17.2671
I20260812 08:01:37.837251 10472 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 17.2673
I20260812 08:01:37.837205 11457 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Connected to new peer: Peer: permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2669, Last known committed idx: 2550, Time since last communication: 0.000s
W20260812 08:01:37.837309 10082 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Aborted: Op aborted by new leader
I20260812 08:01:37.837396 10472 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 17.2674
I20260812 08:01:37.837409 10082 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59446 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2657, attempt_no=0}) took 2218 ms. Trace:
I20260812 08:01:37.837445 10082 rpcz_store.cc:276] 0812 08:01:35.619001 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:35.619489 (+   488us) service_pool.cc:224] Handling call
0812 08:01:37.837400 (+2217911us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:35.619789 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:35.619860 (+    71us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:35.619863 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:35.619865 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:35.619961 (+    96us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:35.619967 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:35.619970 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:35.620017 (+    47us) tablet.cc:701] Row locks acquired
0812 08:01:35.620018 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:35.620047 (+    29us) write_op.cc:270] Start()
0812 08:01:35.620066 (+    19us) write_op.cc:276] Timestamp: P: 1786521695620031 usec, L: 0
0812 08:01:35.620068 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:35.621319 (+  1251us) log.cc:844] Serialized 686 byte log entry
0812 08:01:35.621465 (+   146us) spinlock_profiling.cc:243] Waited 884 us on lock 0x55b31c7d92d4. stack: 00007f9f3e4bfd35 00007f9f3e4c0325 00007f9f3dfcc82d 000055b3045c6b18 000055b3045c6b6b 000055b3045ca9f2 00007f9f4050c3bc 00007f9f40ac1367 00007f9f40ac04cc 00007f9f40abfd9a 00007f9f40ac572a 000055b30465644d 00007f9f3e4e3326 00007f9f3e4e3c4e 00007f9f3e4e539a 000055b30465644d
0812 08:01:37.837285 (+2215820us) op_driver.cc:464] REPLICATION: finished
0812 08:01:37.837290 (+     5us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 08:01:37.837293 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:37.837301 (+     8us) write_op.cc:454] Released schema lock
0812 08:01:37.837306 (+     5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":53,"prepare.run_cpu_time_us":627,"prepare.run_wall_time_us":1703,"replication_time_us":2217189,"spinlock_wait_cycles":2033920}]]}
I20260812 08:01:37.837705 10082 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 17.2672
I20260812 08:01:37.837761 11440 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Connected to new peer: Peer: permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2669, Last known committed idx: 2580, Time since last communication: 0.000s
W20260812 08:01:37.837816 10082 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Aborted: Op aborted by new leader
I20260812 08:01:37.837914 10082 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59446 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2658, attempt_no=0}) took 2213 ms. Trace:
I20260812 08:01:37.837946 10082 rpcz_store.cc:276] 0812 08:01:35.623966 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:35.624034 (+    68us) service_pool.cc:224] Handling call
0812 08:01:37.837906 (+2213872us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:35.624593 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:35.624668 (+    75us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:35.624671 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:35.624672 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:35.624759 (+    87us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:35.624766 (+     7us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:35.624769 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:35.624831 (+    62us) tablet.cc:701] Row locks acquired
0812 08:01:35.624833 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:35.624861 (+    28us) write_op.cc:270] Start()
0812 08:01:35.624881 (+    20us) write_op.cc:276] Timestamp: P: 1786521695624843 usec, L: 0
0812 08:01:35.624882 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:01:35.625018 (+   136us) log.cc:844] Serialized 696 byte log entry
0812 08:01:37.837779 (+2212761us) op_driver.cc:464] REPLICATION: finished
0812 08:01:37.837784 (+     5us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 08:01:37.837787 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:37.837806 (+    19us) write_op.cc:454] Released schema lock
0812 08:01:37.837812 (+     6us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":274,"prepare.run_cpu_time_us":478,"prepare.run_wall_time_us":470,"replication_time_us":2212866}]]}
I20260812 08:01:37.838173 10082 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 17.2673
I20260812 08:01:37.838210  8224 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 18 FOLLOWER]: Refusing update from remote peer 56b60424ed9f4a079689ad82a318d29a: Log matching property violated. Preceding OpId in replica: term: 17 index: 2625. Preceding OpId from leader: term: 18 index: 2670. (index mismatch)
I20260812 08:01:37.838177 11441 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2669, Last known committed idx: 2666, Time since last communication: 0.000s
I20260812 08:01:37.838270 11441 consensus_queue.cc:1243] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Peer 1664e32d3cd0473592085aa522379f22 log is divergent from this leader: its last log entry 17.2669 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2666
W20260812 08:01:37.838284 10082 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Aborted: Op aborted by new leader
I20260812 08:01:37.838385 10082 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59446 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2660, attempt_no=0}) took 2214 ms. Trace:
I20260812 08:01:37.838419 11455 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2669, Last known committed idx: 2666, Time since last communication: 0.000s
I20260812 08:01:37.838513 11455 consensus_queue.cc:1243] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Peer 3ae3f7a6e9f943e996a6dcea504878d0 log is divergent from this leader: its last log entry 17.2669 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2666
I20260812 08:01:37.838424 10082 rpcz_store.cc:276] 0812 08:01:35.624117 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:35.624153 (+    36us) service_pool.cc:224] Handling call
0812 08:01:37.838377 (+2214224us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:35.625374 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:35.625440 (+    66us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:35.625444 (+     4us) write_op.cc:435] Acquired schema lock
0812 08:01:35.625445 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:35.625520 (+    75us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:35.625525 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:35.625528 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:35.625583 (+    55us) tablet.cc:701] Row locks acquired
0812 08:01:35.625585 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:35.625612 (+    27us) write_op.cc:270] Start()
0812 08:01:35.625631 (+    19us) write_op.cc:276] Timestamp: P: 1786521695625594 usec, L: 0
0812 08:01:35.625633 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:35.625735 (+   102us) log.cc:844] Serialized 696 byte log entry
0812 08:01:37.838246 (+2212511us) op_driver.cc:464] REPLICATION: finished
0812 08:01:37.838252 (+     6us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 08:01:37.838255 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:37.838274 (+    19us) write_op.cc:454] Released schema lock
0812 08:01:37.838279 (+     5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":969,"prepare.run_cpu_time_us":412,"prepare.run_wall_time_us":411,"replication_time_us":2212581}]]}
I20260812 08:01:37.838666 10082 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 17.2674
W20260812 08:01:37.838764 10082 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Aborted: Op aborted by new leader
I20260812 08:01:37.838847 10082 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59446 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2659, attempt_no=0}) took 2214 ms. Trace:
I20260812 08:01:37.838877 10082 rpcz_store.cc:276] 0812 08:01:35.624053 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:35.624347 (+   294us) service_pool.cc:224] Handling call
0812 08:01:37.838840 (+2214493us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:35.625809 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:35.625879 (+    70us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:35.625883 (+     4us) write_op.cc:435] Acquired schema lock
0812 08:01:35.625884 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:35.625963 (+    79us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:35.625969 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:35.625973 (+     4us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:35.626029 (+    56us) tablet.cc:701] Row locks acquired
0812 08:01:35.626030 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:35.626063 (+    33us) write_op.cc:270] Start()
0812 08:01:35.626081 (+    18us) write_op.cc:276] Timestamp: P: 1786521695626038 usec, L: 0
0812 08:01:35.626083 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:35.626183 (+   100us) log.cc:844] Serialized 696 byte log entry
0812 08:01:37.838733 (+2212550us) op_driver.cc:464] REPLICATION: finished
0812 08:01:37.838738 (+     5us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 08:01:37.838741 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:37.838758 (+    17us) write_op.cc:454] Released schema lock
0812 08:01:37.838762 (+     4us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":1255,"prepare.run_cpu_time_us":409,"prepare.run_wall_time_us":408,"replication_time_us":2212622}]]}
I20260812 08:01:37.838953  9808 raft_consensus.cc:1217] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 18 FOLLOWER]: Deduplicated request from leader. Original: 17.2666->[17.2667-18.2670]   Dedup: 17.2668->[18.2669-18.2670]
I20260812 08:01:37.839027  9808 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting all ops after (but not including) 2668
I20260812 08:01:37.839051  9808 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting uncommitted WRITE_OP operation due to leader change: 17.2669
I20260812 08:01:37.839275 11455 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2669, Last known committed idx: 2625, Time since last communication: 0.000s
I20260812 08:01:37.839960 11455 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Connected to new peer: Peer: permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2669, Last known committed idx: 2668, Time since last communication: 0.000s
I20260812 08:01:37.840049 11455 consensus_queue.cc:1243] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Peer faafbc72cbd444ff90977a2cdb091b72 log is divergent from this leader: its last log entry 17.2669 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2668
I20260812 08:01:37.841374 10472 raft_consensus.cc:1217] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 18 FOLLOWER]: Deduplicated request from leader. Original: 17.2666->[17.2667-18.2670]   Dedup: 17.2668->[18.2669-18.2670]
I20260812 08:01:37.841472 10472 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting all ops after (but not including) 2668
I20260812 08:01:37.841495 10472 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 17.2669
I20260812 08:01:37.842671 10082 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting all ops after (but not including) 2668
I20260812 08:01:37.842725 10082 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 17.2669
W20260812 08:01:37.842828 10082 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Aborted: Op aborted by new leader
I20260812 08:01:37.842939 10082 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59446 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2655, attempt_no=0}) took 2224 ms. Trace:
I20260812 08:01:37.842998 10082 rpcz_store.cc:276] 0812 08:01:35.618223 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:35.618259 (+    36us) service_pool.cc:224] Handling call
0812 08:01:37.842930 (+2224671us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:35.618612 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:35.618695 (+    83us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:35.618698 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:35.618700 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:35.618784 (+    84us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:35.618790 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:35.618793 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:35.618854 (+    61us) tablet.cc:701] Row locks acquired
0812 08:01:35.618855 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:35.618878 (+    23us) write_op.cc:270] Start()
0812 08:01:35.618899 (+    21us) write_op.cc:276] Timestamp: P: 1786521695618862 usec, L: 0
0812 08:01:35.618900 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:01:35.619004 (+   104us) log.cc:844] Serialized 696 byte log entry
0812 08:01:37.842789 (+2223785us) op_driver.cc:464] REPLICATION: finished
0812 08:01:37.842796 (+     7us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 08:01:37.842799 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:37.842819 (+    20us) write_op.cc:454] Released schema lock
0812 08:01:37.842825 (+     6us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":52,"prepare.run_cpu_time_us":445,"prepare.run_wall_time_us":446,"replication_time_us":2223856}]]}
I20260812 08:01:39.655718 11534 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:39.655876 11534 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:39.656672 11534 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:39.657102  9809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 19 candidate_status { last_received { term: 18 index: 2732 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:39.657102 10082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 19 candidate_status { last_received { term: 18 index: 2732 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:39.657272 10462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 19 candidate_status { last_received { term: 18 index: 2732 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:39.657286  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 19 candidate_status { last_received { term: 18 index: 2732 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:39.657754  7287 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 16901c5559a748a9aa770d73e254099c; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:39.657764  8006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 19 candidate_status { last_received { term: 18 index: 2732 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:39.657939 11534 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260812 08:01:39.886727 11535 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:39.886904 11535 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:39.887187 11540 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:39.887300 11540 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:39.887843 11535 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:39.888041 11540 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:39.888072 10082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 19 candidate_status { last_received { term: 18 index: 2818 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:39.888083  9809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 19 candidate_status { last_received { term: 18 index: 2818 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:39.888208 10082 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 18.
I20260812 08:01:39.888486  9808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 19 candidate_status { last_received { term: 18 index: 2818 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:39.888742  8006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 19 candidate_status { last_received { term: 18 index: 2818 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:39.888765  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 19 candidate_status { last_received { term: 18 index: 2818 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:39.889099  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 19 candidate_status { last_received { term: 18 index: 2818 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:39.889307 10462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 19 candidate_status { last_received { term: 18 index: 2818 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:39.889403  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 19 candidate_status { last_received { term: 18 index: 2818 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:39.889431 10462 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 18.
I20260812 08:01:39.889578  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 19 candidate_status { last_received { term: 18 index: 2818 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:39.889578  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 19 candidate_status { last_received { term: 18 index: 2818 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:39.889730  7381 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 18.
I20260812 08:01:39.890102 10367 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 3ae3f7a6e9f943e996a6dcea504878d0, faafbc72cbd444ff90977a2cdb091b72; no voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df
I20260812 08:01:39.890301 11535 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260812 08:01:40.224588 11541 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:40.224737 11541 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:40.225543 11541 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183)
I20260812 08:01:40.225967 10462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 19 candidate_status { last_received { term: 18 index: 2705 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:40.226020 10082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 19 candidate_status { last_received { term: 18 index: 2705 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:40.226020  9808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 19 candidate_status { last_received { term: 18 index: 2705 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:40.226157 10462 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df for term 19 because replica has last-logged OpId of term: 18 index: 2818, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 2705.
I20260812 08:01:40.226181 10082 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df for term 19 because replica has last-logged OpId of term: 18 index: 2818, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 2705.
I20260812 08:01:40.226210  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 19 candidate_status { last_received { term: 18 index: 2705 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:40.226389  7381 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df for term 19 because replica has last-logged OpId of term: 18 index: 2732, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 2705.
I20260812 08:01:40.226679  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 19 candidate_status { last_received { term: 18 index: 2705 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:40.226773  7898 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:40.226867  8225 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df for term 19 because replica has last-logged OpId of term: 18 index: 2770, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 2705.
I20260812 08:01:40.226972 11541 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260812 08:01:40.227330 11542 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:40.227416 11542 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:40.228015 11542 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:40.228256  9808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 2770 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:40.228304  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 2770 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:40.228331 10462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 2770 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:40.228411  7381 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6801d45ca6614156906e5d571ebcb8a6 in term 18.
I20260812 08:01:40.228456 10462 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6801d45ca6614156906e5d571ebcb8a6 for term 19 because replica has last-logged OpId of term: 18 index: 2818, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 2770.
I20260812 08:01:40.228502  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 2770 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:40.228623  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6801d45ca6614156906e5d571ebcb8a6 in term 18.
I20260812 08:01:40.228811 10082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 2770 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:40.228991 10082 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6801d45ca6614156906e5d571ebcb8a6 for term 19 because replica has last-logged OpId of term: 18 index: 2818, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 2770.
I20260812 08:01:40.230950 11543 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:40.231056 11543 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:40.231797 11543 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:40.231848 10082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 19 candidate_status { last_received { term: 18 index: 2812 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:40.231981 10082 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1664e32d3cd0473592085aa522379f22 for term 19 because replica has last-logged OpId of term: 18 index: 2818, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 2812.
I20260812 08:01:40.231957  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 19 candidate_status { last_received { term: 18 index: 2812 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:40.232025  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 19 candidate_status { last_received { term: 18 index: 2812 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:40.232069  7381 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 18.
I20260812 08:01:40.232122  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 18.
I20260812 08:01:40.232194  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 19 candidate_status { last_received { term: 18 index: 2812 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:40.232297  8225 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 18.
I20260812 08:01:40.232574  9708 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:40.232636 10462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 19 candidate_status { last_received { term: 18 index: 2812 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:40.232739 11543 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260812 08:01:40.232785 10462 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1664e32d3cd0473592085aa522379f22 for term 19 because replica has last-logged OpId of term: 18 index: 2818, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 2812.
I20260812 08:01:40.232798 11543 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:40.232827 11543 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 18 FOLLOWER]: Advancing to term 19
I20260812 08:01:40.233771 11543 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:40.234552 11543 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 19 election: Requested vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:40.234575  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 19 candidate_status { last_received { term: 18 index: 2812 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
I20260812 08:01:40.234642  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 19 candidate_status { last_received { term: 18 index: 2812 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:01:40.234694  8225 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 18 FOLLOWER]: Advancing to term 19
I20260812 08:01:40.234673 10082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 19 candidate_status { last_received { term: 18 index: 2812 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72"
I20260812 08:01:40.234730  7381 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 18 FOLLOWER]: Advancing to term 19
I20260812 08:01:40.234772 10082 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 18 FOLLOWER]: Advancing to term 19
I20260812 08:01:40.235560 10082 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 1664e32d3cd0473592085aa522379f22 for term 19 because replica has last-logged OpId of term: 18 index: 2818, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 2812.
I20260812 08:01:40.235721  8225 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 19.
I20260812 08:01:40.235734  7381 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 19.
I20260812 08:01:40.235908 10462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 19 candidate_status { last_received { term: 18 index: 2812 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0"
I20260812 08:01:40.236027 10462 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 18 FOLLOWER]: Advancing to term 19
I20260812 08:01:40.236445  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 19 candidate_status { last_received { term: 18 index: 2812 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:01:40.236557  8004 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 18 FOLLOWER]: Advancing to term 19
I20260812 08:01:40.236864 10462 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 1664e32d3cd0473592085aa522379f22 for term 19 because replica has last-logged OpId of term: 18 index: 2818, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 2812.
I20260812 08:01:40.237480  8004 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 19.
I20260812 08:01:40.237769  9706 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 3ae3f7a6e9f943e996a6dcea504878d0, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:40.237988 11543 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 19 FOLLOWER]: Leader election won for term 19
I20260812 08:01:40.238188 11543 raft_consensus.cc:697] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 19 LEADER]: Becoming Leader. State: Replica: 1664e32d3cd0473592085aa522379f22, State: Running, Role: LEADER
I20260812 08:01:40.238536 11543 consensus_queue.cc:237] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2810, Committed index: 2810, Last appended: 18.2812, Last appended by leader: 2812, Current term: 19, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:40.241384  6748 catalog_manager.cc:5719] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 reported cstate change: term changed from 18 to 19, leader changed from 56b60424ed9f4a079689ad82a318d29a (127.5.142.130) to 1664e32d3cd0473592085aa522379f22 (127.5.142.133). New cstate: current_term: 19 leader_uuid: "1664e32d3cd0473592085aa522379f22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } health_report { overall_health: UNKNOWN } } }
W20260812 08:01:40.573654 10631 connection.cc:256] RPC call timeout handler was delayed by -1.5191165830474347s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 08:01:40.574419 10623 connection.cc:256] RPC call timeout handler was delayed by -1.6291841859929264s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 08:01:40.574905 10624 connection.cc:256] RPC call timeout handler was delayed by -1.2945723161101341s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 08:01:40.575125 10629 connection.cc:256] RPC call timeout handler was delayed by -1.4063511490821838s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 08:01:40.575903 10730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 2770 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:40.576491  8116 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:40.576862 11542 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260812 08:01:40.578526 10723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 19 candidate_status { last_received { term: 18 index: 2705 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:40.578912 10082 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 19 FOLLOWER]: Rejecting Update request from peer 56b60424ed9f4a079689ad82a318d29a for earlier term 18. Current term is 19. Ops: []
I20260812 08:01:40.579161 10727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 19 candidate_status { last_received { term: 18 index: 2818 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:40.579183  7381 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 19 FOLLOWER]: Rejecting Update request from peer 56b60424ed9f4a079689ad82a318d29a for earlier term 18. Current term is 19. Ops: [18.2733-18.2818]
I20260812 08:01:40.579358 10726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 19 candidate_status { last_received { term: 18 index: 2812 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:40.579691 10728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 19 candidate_status { last_received { term: 18 index: 2812 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a"
I20260812 08:01:40.579983 10738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 19 candidate_status { last_received { term: 18 index: 2732 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:40.580072 11440 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 }, Status: INVALID_TERM, Last received: 18.2818, Next index: 2819, Last known committed idx: 2810, Time since last communication: 0.000s
I20260812 08:01:40.580224 11440 raft_consensus.cc:3055] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 18 LEADER]: Stepping down as leader of term 18
I20260812 08:01:40.580263 11440 raft_consensus.cc:740] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 56b60424ed9f4a079689ad82a318d29a, State: Running, Role: LEADER
I20260812 08:01:40.580340 10734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 19 candidate_status { last_received { term: 18 index: 2818 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:40.580391 11440 consensus_queue.cc:260] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2812, Committed index: 2812, Last appended: 18.2818, Last appended by leader: 2818, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:40.580570 11440 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 18 FOLLOWER]: Advancing to term 19
I20260812 08:01:40.581200 11482 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:58954 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2795, attempt_no=0}) took 2211 ms. Trace:
I20260812 08:01:40.581264 11482 rpcz_store.cc:276] 0812 08:01:38.369417 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:38.370941 (+  1524us) service_pool.cc:224] Handling call
0812 08:01:40.581176 (+2210235us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:38.376478 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:38.376549 (+    71us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:38.376553 (+     4us) write_op.cc:435] Acquired schema lock
0812 08:01:38.376554 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:38.376650 (+    96us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:38.376656 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:38.376658 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:38.376718 (+    60us) tablet.cc:701] Row locks acquired
0812 08:01:38.376719 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:38.376748 (+    29us) write_op.cc:270] Start()
0812 08:01:38.376768 (+    20us) write_op.cc:276] Timestamp: P: 1786521698376704 usec, L: 0
0812 08:01:38.376769 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:01:38.376874 (+   105us) log.cc:844] Serialized 696 byte log entry
0812 08:01:40.578038 (+2201164us) op_driver.cc:464] REPLICATION: finished
0812 08:01:40.580731 (+  2693us) write_op.cc:301] APPLY: starting
0812 08:01:40.580744 (+    13us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:40.580774 (+    30us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:40.580785 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:40.580983 (+   198us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:40.581011 (+    28us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:40.581015 (+     4us) write_op.cc:312] APPLY: finished
0812 08:01:40.581075 (+    60us) log.cc:844] Serialized 77 byte log entry
0812 08:01:40.581112 (+    37us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:40.581123 (+    11us) write_op.cc:454] Released schema lock
0812 08:01:40.581128 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2652,"num_ops":10,"prepare.queue_time_us":5297,"prepare.run_cpu_time_us":467,"prepare.run_wall_time_us":466,"replication_time_us":2201259,"thread_start_us":94,"threads_started":1}]]}
I20260812 08:01:40.581475  8004 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 19 FOLLOWER]: Rejecting Update request from peer 56b60424ed9f4a079689ad82a318d29a for earlier term 18. Current term is 19. Ops: [18.2706-18.2818]
I20260812 08:01:40.581779 10734 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 19.
I20260812 08:01:40.581872  9808 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 19 LEADER]: Rejecting Update request from peer 56b60424ed9f4a079689ad82a318d29a for earlier term 18. Current term is 19. Ops: [18.2813-18.2818]
I20260812 08:01:40.582554 10462 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 19 FOLLOWER]: Rejecting Update request from peer 56b60424ed9f4a079689ad82a318d29a for earlier term 18. Current term is 19. Ops: []
I20260812 08:01:40.580577 11552 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:58954 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2797, attempt_no=0}) took 2211 ms. Trace:
I20260812 08:01:40.582633 11552 rpcz_store.cc:276] 0812 08:01:38.369537 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:38.370115 (+   578us) service_pool.cc:224] Handling call
0812 08:01:40.580556 (+2210441us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:38.376021 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:38.376100 (+    79us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:38.376103 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:38.376104 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:38.376190 (+    86us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:38.376196 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:38.376199 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:38.376261 (+    62us) tablet.cc:701] Row locks acquired
0812 08:01:38.376263 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:38.376293 (+    30us) write_op.cc:270] Start()
0812 08:01:38.376311 (+    18us) write_op.cc:276] Timestamp: P: 1786521698376247 usec, L: 0
0812 08:01:38.376313 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:38.376403 (+    90us) log.cc:844] Serialized 696 byte log entry
0812 08:01:40.577994 (+2201591us) op_driver.cc:464] REPLICATION: finished
0812 08:01:40.579854 (+  1860us) write_op.cc:301] APPLY: starting
0812 08:01:40.579875 (+    21us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:40.579922 (+    47us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:40.579935 (+    13us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:40.580164 (+   229us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:40.580198 (+    34us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:40.580203 (+     5us) write_op.cc:312] APPLY: finished
0812 08:01:40.580287 (+    84us) log.cc:844] Serialized 77 byte log entry
0812 08:01:40.580458 (+   171us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:40.580478 (+    20us) write_op.cc:454] Released schema lock
0812 08:01:40.580486 (+     8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1803,"num_ops":10,"prepare.queue_time_us":5675,"prepare.run_cpu_time_us":433,"prepare.run_wall_time_us":432,"replication_time_us":2201650,"thread_start_us":76,"threads_started":1,"wal-append.queue_time_us":2124}]]}
I20260812 08:01:40.584127  8225 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 19 FOLLOWER]: Rejecting Update request from peer 56b60424ed9f4a079689ad82a318d29a for earlier term 18. Current term is 19. Ops: [18.2771-18.2818]
W20260812 08:01:40.584834 10662 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58954: Illegal state: replica 56b60424ed9f4a079689ad82a318d29a is not leader of this config: current role FOLLOWER
I20260812 08:01:40.584985  9966 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, faafbc72cbd444ff90977a2cdb091b72; no voters: 1664e32d3cd0473592085aa522379f22, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df
I20260812 08:01:40.585290 11557 raft_consensus.cc:2764] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
W20260812 08:01:40.584846 10664 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58954: Illegal state: replica 56b60424ed9f4a079689ad82a318d29a is not leader of this config: current role FOLLOWER
W20260812 08:01:40.587525 10412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:40.591629 10412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:40.594852 10005 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Illegal state: replica faafbc72cbd444ff90977a2cdb091b72 is not leader of this config: current role FOLLOWER
W20260812 08:01:40.595839 10005 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Illegal state: replica faafbc72cbd444ff90977a2cdb091b72 is not leader of this config: current role FOLLOWER
I20260812 08:01:40.601182  8225 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 19 FOLLOWER]: Refusing update from remote peer 1664e32d3cd0473592085aa522379f22: Log matching property violated. Preceding OpId in replica: term: 18 index: 2770. Preceding OpId from leader: term: 19 index: 2814. (index mismatch)
I20260812 08:01:40.601253  7381 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 19 FOLLOWER]: Refusing update from remote peer 1664e32d3cd0473592085aa522379f22: Log matching property violated. Preceding OpId in replica: term: 18 index: 2732. Preceding OpId from leader: term: 19 index: 2814. (index mismatch)
I20260812 08:01:40.601385  8004 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 19 FOLLOWER]: Refusing update from remote peer 1664e32d3cd0473592085aa522379f22: Log matching property violated. Preceding OpId in replica: term: 18 index: 2705. Preceding OpId from leader: term: 19 index: 2814. (index mismatch)
I20260812 08:01:40.601740 11543 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2813, Last known committed idx: 2726, Time since last communication: 0.000s
I20260812 08:01:40.602005 10082 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 19 FOLLOWER]: Refusing update from remote peer 1664e32d3cd0473592085aa522379f22: Log matching property violated. Preceding OpId in replica: term: 18 index: 2818. Preceding OpId from leader: term: 19 index: 2814. (term mismatch)
I20260812 08:01:40.602056 10082 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting all ops after (but not including) 2813
I20260812 08:01:40.602035 10725 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 19 FOLLOWER]: Refusing update from remote peer 1664e32d3cd0473592085aa522379f22: Log matching property violated. Preceding OpId in replica: term: 18 index: 2818. Preceding OpId from leader: term: 19 index: 2814. (term mismatch)
I20260812 08:01:40.602021 10462 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 19 FOLLOWER]: Refusing update from remote peer 1664e32d3cd0473592085aa522379f22: Log matching property violated. Preceding OpId in replica: term: 18 index: 2818. Preceding OpId from leader: term: 19 index: 2814. (term mismatch)
I20260812 08:01:40.602077 10082 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 18.2814
I20260812 08:01:40.602090 10725 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting all ops after (but not including) 2813
I20260812 08:01:40.602095 10462 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting all ops after (but not including) 2813
I20260812 08:01:40.602115 10725 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting uncommitted WRITE_OP operation due to leader change: 18.2814
I20260812 08:01:40.602120 10462 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 18.2814
W20260812 08:01:40.602260 10725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58954: Aborted: Op aborted by new leader
I20260812 08:01:40.602273 10082 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 18.2815
I20260812 08:01:40.602336 10462 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 18.2815
I20260812 08:01:40.602396 10725 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:58954 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2799, attempt_no=0}) took 2230 ms. Trace:
I20260812 08:01:40.602419 10082 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 18.2816
I20260812 08:01:40.602488 10462 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 18.2816
I20260812 08:01:40.602545 10082 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 18.2817
I20260812 08:01:40.602612 10462 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 18.2817
I20260812 08:01:40.602449 10725 rpcz_store.cc:276] 0812 08:01:38.371778 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:38.371811 (+    33us) service_pool.cc:224] Handling call
0812 08:01:40.602385 (+2230574us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:38.380772 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:38.380850 (+    78us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:38.380853 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:38.380855 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:38.380967 (+   112us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:38.380973 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:38.380976 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:38.381036 (+    60us) tablet.cc:701] Row locks acquired
0812 08:01:38.381037 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:38.381068 (+    31us) write_op.cc:270] Start()
0812 08:01:38.381088 (+    20us) write_op.cc:276] Timestamp: P: 1786521698381022 usec, L: 0
0812 08:01:38.381090 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:38.381197 (+   107us) log.cc:844] Serialized 696 byte log entry
0812 08:01:40.602188 (+2220991us) op_driver.cc:464] REPLICATION: finished
0812 08:01:40.602218 (+    30us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 08:01:40.602222 (+     4us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:40.602249 (+    27us) write_op.cc:454] Released schema lock
0812 08:01:40.602256 (+     7us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":8686,"prepare.run_cpu_time_us":476,"prepare.run_wall_time_us":668,"replication_time_us":2221054}]]}
I20260812 08:01:40.602659 10082 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 18.2818
I20260812 08:01:40.602746 10462 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 18.2818
I20260812 08:01:40.602764 10725 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting uncommitted WRITE_OP operation due to leader change: 18.2815
W20260812 08:01:40.602866 10725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58954: Aborted: Op aborted by new leader
I20260812 08:01:40.602984 10725 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:58954 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2800, attempt_no=0}) took 2230 ms. Trace:
I20260812 08:01:40.603022 10725 rpcz_store.cc:276] 0812 08:01:38.372167 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:38.372200 (+    33us) service_pool.cc:224] Handling call
0812 08:01:40.602975 (+2230775us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:38.381465 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:38.381536 (+    71us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:38.381539 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:38.381540 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:38.381618 (+    78us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:38.381624 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:38.381627 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:38.382397 (+   770us) tablet.cc:701] Row locks acquired
0812 08:01:38.382399 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:38.382434 (+    35us) write_op.cc:270] Start()
0812 08:01:38.382456 (+    22us) write_op.cc:276] Timestamp: P: 1786521698382388 usec, L: 0
0812 08:01:38.382458 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:38.382564 (+   106us) log.cc:844] Serialized 686 byte log entry
0812 08:01:40.602829 (+2220265us) op_driver.cc:464] REPLICATION: finished
0812 08:01:40.602834 (+     5us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 08:01:40.602837 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:40.602857 (+    20us) write_op.cc:454] Released schema lock
0812 08:01:40.602863 (+     6us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":9048,"prepare.run_cpu_time_us":462,"prepare.run_wall_time_us":1342,"replication_time_us":2220342}]]}
I20260812 08:01:40.603122 11559 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2813, Last known committed idx: 2764, Time since last communication: 0.000s
I20260812 08:01:40.603260 10725 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting uncommitted WRITE_OP operation due to leader change: 18.2816
W20260812 08:01:40.603374 10725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58954: Aborted: Op aborted by new leader
I20260812 08:01:40.603474 10725 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:58954 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2801, attempt_no=0}) took 2223 ms. Trace:
I20260812 08:01:40.603518 10725 rpcz_store.cc:276] 0812 08:01:38.379854 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:38.379913 (+    59us) service_pool.cc:224] Handling call
0812 08:01:40.603466 (+2223553us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:38.382830 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:38.382902 (+    72us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:38.382905 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:38.382906 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:38.382986 (+    80us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:38.382992 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:38.382995 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:38.383059 (+    64us) tablet.cc:701] Row locks acquired
0812 08:01:38.383061 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:38.383093 (+    32us) write_op.cc:270] Start()
0812 08:01:38.383112 (+    19us) write_op.cc:276] Timestamp: P: 1786521698383046 usec, L: 0
0812 08:01:38.383114 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:38.383217 (+   103us) log.cc:844] Serialized 696 byte log entry
0812 08:01:40.603336 (+2220119us) op_driver.cc:464] REPLICATION: finished
0812 08:01:40.603343 (+     7us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 08:01:40.603346 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:40.603365 (+    19us) write_op.cc:454] Released schema lock
0812 08:01:40.603371 (+     6us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":2630,"prepare.run_cpu_time_us":429,"prepare.run_wall_time_us":606,"replication_time_us":2220189}]]}
I20260812 08:01:40.603650 11545 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2813, Last known committed idx: 2810, Time since last communication: 0.000s
I20260812 08:01:40.603749 11545 consensus_queue.cc:1243] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Peer 3ae3f7a6e9f943e996a6dcea504878d0 log is divergent from this leader: its last log entry 18.2813 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2810
I20260812 08:01:40.603782 10725 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting uncommitted WRITE_OP operation due to leader change: 18.2817
W20260812 08:01:40.603890 10725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58954: Aborted: Op aborted by new leader
I20260812 08:01:40.603909 11559 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2813, Last known committed idx: 2810, Time since last communication: 0.000s
I20260812 08:01:40.603991 10725 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:58954 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2802, attempt_no=0}) took 2220 ms. Trace:
I20260812 08:01:40.604009 11559 consensus_queue.cc:1243] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Peer faafbc72cbd444ff90977a2cdb091b72 log is divergent from this leader: its last log entry 18.2813 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2810
I20260812 08:01:40.604028 10725 rpcz_store.cc:276] 0812 08:01:38.383159 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:38.383287 (+   128us) service_pool.cc:224] Handling call
0812 08:01:40.603981 (+2220694us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:38.383587 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:38.383662 (+    75us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:38.383665 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:38.383666 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:38.383729 (+    63us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:38.383733 (+     4us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:38.383735 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:38.383777 (+    42us) tablet.cc:701] Row locks acquired
0812 08:01:38.383778 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:38.383801 (+    23us) write_op.cc:270] Start()
0812 08:01:38.383815 (+    14us) write_op.cc:276] Timestamp: P: 1786521698383758 usec, L: 0
0812 08:01:38.383816 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:01:38.383898 (+    82us) log.cc:844] Serialized 696 byte log entry
0812 08:01:40.603852 (+2219954us) op_driver.cc:464] REPLICATION: finished
0812 08:01:40.603857 (+     5us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 08:01:40.603861 (+     4us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:40.603881 (+    20us) write_op.cc:454] Released schema lock
0812 08:01:40.603887 (+     6us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":39,"prepare.run_cpu_time_us":359,"prepare.run_wall_time_us":358,"replication_time_us":2220003}]]}
I20260812 08:01:40.604265 10725 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting uncommitted WRITE_OP operation due to leader change: 18.2818
W20260812 08:01:40.604372 10725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58954: Aborted: Op aborted by new leader
I20260812 08:01:40.604478 10725 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:58954 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2803, attempt_no=0}) took 2221 ms. Trace:
I20260812 08:01:40.604451 11562 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2813, Last known committed idx: 2704, Time since last communication: 0.000s
I20260812 08:01:40.604525 10725 rpcz_store.cc:276] 0812 08:01:38.383234 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:38.383409 (+   175us) service_pool.cc:224] Handling call
0812 08:01:40.604469 (+2221060us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:38.383973 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:38.384039 (+    66us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:38.384042 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:38.384044 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:38.384122 (+    78us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:38.384127 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:38.384130 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:38.384188 (+    58us) tablet.cc:701] Row locks acquired
0812 08:01:38.384189 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:38.384213 (+    24us) write_op.cc:270] Start()
0812 08:01:38.384227 (+    14us) write_op.cc:276] Timestamp: P: 1786521698384169 usec, L: 0
0812 08:01:38.384229 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:38.384311 (+    82us) log.cc:844] Serialized 696 byte log entry
0812 08:01:40.604337 (+2220026us) op_driver.cc:464] REPLICATION: finished
0812 08:01:40.604342 (+     5us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 08:01:40.604345 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:40.604364 (+    19us) write_op.cc:454] Released schema lock
0812 08:01:40.604369 (+     5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":367,"prepare.run_cpu_time_us":385,"prepare.run_wall_time_us":560,"replication_time_us":2220079}]]}
I20260812 08:01:40.605603 10462 raft_consensus.cc:1217] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 19 FOLLOWER]: Deduplicated request from leader. Original: 18.2810->[18.2811-19.2814]   Dedup: 18.2812->[19.2813-19.2814]
I20260812 08:01:40.605696 10462 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting all ops after (but not including) 2812
I20260812 08:01:40.605718 10462 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 18.2813
I20260812 08:01:40.606773 11562 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2813, Last known committed idx: 2812, Time since last communication: 0.000s
I20260812 08:01:40.606871 11562 consensus_queue.cc:1243] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Peer 56b60424ed9f4a079689ad82a318d29a log is divergent from this leader: its last log entry 18.2813 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2812
I20260812 08:01:40.607514 10725 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting all ops after (but not including) 2812
I20260812 08:01:40.607565 10725 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting uncommitted WRITE_OP operation due to leader change: 18.2813
I20260812 08:01:40.607589 10082 raft_consensus.cc:1217] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 19 FOLLOWER]: Deduplicated request from leader. Original: 18.2810->[18.2811-19.2814]   Dedup: 18.2812->[19.2813-19.2814]
I20260812 08:01:40.607666 10082 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting all ops after (but not including) 2812
I20260812 08:01:40.607692 10082 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 18.2813
W20260812 08:01:40.607683 10725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58954: Aborted: Op aborted by new leader
I20260812 08:01:40.607791 10725 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:58954 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2798, attempt_no=0}) took 2236 ms. Trace:
I20260812 08:01:40.607832 10725 rpcz_store.cc:276] 0812 08:01:38.371423 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:38.371461 (+    38us) service_pool.cc:224] Handling call
0812 08:01:40.607782 (+2236321us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:38.376976 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:38.377048 (+    72us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:38.377051 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:38.377052 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:38.377135 (+    83us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:38.377141 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:38.379959 (+  2818us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:38.380036 (+    77us) tablet.cc:701] Row locks acquired
0812 08:01:38.380038 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:38.380068 (+    30us) write_op.cc:270] Start()
0812 08:01:38.380089 (+    21us) write_op.cc:276] Timestamp: P: 1786521698380023 usec, L: 0
0812 08:01:38.380090 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:01:38.380192 (+   102us) log.cc:844] Serialized 686 byte log entry
0812 08:01:40.607643 (+2227451us) op_driver.cc:464] REPLICATION: finished
0812 08:01:40.607649 (+     6us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 08:01:40.607653 (+     4us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:40.607674 (+    21us) write_op.cc:454] Released schema lock
0812 08:01:40.607679 (+     5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"mutex_wait_us":17,"num_ops":10,"prepare.queue_time_us":5281,"prepare.run_cpu_time_us":512,"prepare.run_wall_time_us":3766,"raft.queue_time_us":324,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":262,"replication_time_us":2227519,"spinlock_wait_cycles":30080}]]}
I20260812 08:01:42.727586 11627 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1664e32d3cd0473592085aa522379f22)
I20260812 08:01:42.727738 11627 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:42.728574 11627 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:42.729058  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 2969 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:42.729220 10729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 2969 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:42.729265  8006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 2969 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:42.729413 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 2969 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:42.729568 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 2969 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:42.729939  8118 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 6801d45ca6614156906e5d571ebcb8a6; no voters: 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, 780d7e64a8214418a371f35351b113df
I20260812 08:01:42.730173 11627 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260812 08:01:42.838603 11633 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1664e32d3cd0473592085aa522379f22)
I20260812 08:01:42.838774 11633 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:42.839607 11633 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:42.839943 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 20 candidate_status { last_received { term: 19 index: 3010 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:42.839938 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 20 candidate_status { last_received { term: 19 index: 3010 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:42.840041  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 20 candidate_status { last_received { term: 19 index: 3010 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:42.840308  8224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 20 candidate_status { last_received { term: 19 index: 3010 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:42.840468  8224 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 19.
I20260812 08:01:42.840763  8006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 20 candidate_status { last_received { term: 19 index: 3010 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:42.841205 10623 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6; no voters: 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, 780d7e64a8214418a371f35351b113df, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:42.841401 11633 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260812 08:01:42.843672 11635 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1664e32d3cd0473592085aa522379f22)
I20260812 08:01:42.843787 11635 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:42.843881 11638 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1664e32d3cd0473592085aa522379f22)
I20260812 08:01:42.843993 11638 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:42.844820 11638 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:42.845135 10729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 20 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:42.845269 10729 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 19.
I20260812 08:01:42.845275  8006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 20 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:42.845355  8224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 20 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:42.845465  8224 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 19.
I20260812 08:01:42.845474 10725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 20 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:42.845595 10725 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 19.
I20260812 08:01:42.845625 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 20 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:42.845728 10079 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 19.
I20260812 08:01:42.845722  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 20 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:42.845827  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 20 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:42.845908 11635 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:42.845933  8225 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 19.
I20260812 08:01:42.846235 10364 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72; no voters: 
I20260812 08:01:42.846338 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 20 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:42.846443 10472 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 19.
I20260812 08:01:42.846421  8006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 20 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:42.846520 11635 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260812 08:01:42.846576 11635 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 1664e32d3cd0473592085aa522379f22)
I20260812 08:01:42.846614 11635 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 19 FOLLOWER]: Advancing to term 20
I20260812 08:01:42.846726  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 20 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:42.846758  9969 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72; no voters: 16901c5559a748a9aa770d73e254099c, 780d7e64a8214418a371f35351b113df
I20260812 08:01:42.846918 11638 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260812 08:01:42.846968 11638 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 1664e32d3cd0473592085aa522379f22)
I20260812 08:01:42.847002 11638 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 19 FOLLOWER]: Advancing to term 20
I20260812 08:01:42.847779 11635 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:42.847996 11638 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:42.848618 11638 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 20 election: Requested vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:42.848791 10725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 20 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a"
I20260812 08:01:42.848928 10725 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 19 FOLLOWER]: Advancing to term 20
I20260812 08:01:42.848912 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 20 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72"
I20260812 08:01:42.849086 10079 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ae3f7a6e9f943e996a6dcea504878d0 in current term 20: Already voted for candidate faafbc72cbd444ff90977a2cdb091b72 in this term.
I20260812 08:01:42.849076  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 20 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
I20260812 08:01:42.849177  8225 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 19 FOLLOWER]: Advancing to term 20
I20260812 08:01:42.849169 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 20 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0"
I20260812 08:01:42.849304 10472 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate faafbc72cbd444ff90977a2cdb091b72 in current term 20: Already voted for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in this term.
I20260812 08:01:42.849643  8006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 20 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:01:42.849989 10725 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 20.
I20260812 08:01:42.850070  8224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 20 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
I20260812 08:01:42.850193 10729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 20 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a"
I20260812 08:01:42.850298  8225 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 20.
I20260812 08:01:42.850324 10729 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate faafbc72cbd444ff90977a2cdb091b72 in current term 20: Already voted for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in this term.
I20260812 08:01:42.850370 11635 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 20 election: Requested vote from peers faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:42.850813  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 20 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:01:42.850857  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 20 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:01:42.850906  8006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 20 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:01:42.851286  9969 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72; no voters: 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, 780d7e64a8214418a371f35351b113df
I20260812 08:01:42.851281 10367 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a; no voters: 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:42.851465 11638 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260812 08:01:42.851475 11635 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260812 08:01:43.186195 11639 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1664e32d3cd0473592085aa522379f22)
I20260812 08:01:43.186285 11640 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1664e32d3cd0473592085aa522379f22)
I20260812 08:01:43.186357 11639 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:43.186407 11640 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:43.187645 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 20 candidate_status { last_received { term: 19 index: 2900 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:43.187649 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 20 candidate_status { last_received { term: 19 index: 2900 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:43.187760 10729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 20 candidate_status { last_received { term: 19 index: 2900 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:43.187829 10079 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df in current term 20: Already voted for candidate faafbc72cbd444ff90977a2cdb091b72 in this term.
I20260812 08:01:43.187829 10472 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df in current term 20: Already voted for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in this term.
I20260812 08:01:43.187907 10729 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df in current term 20: Already voted for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in this term.
I20260812 08:01:43.187968 11639 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:43.188043 10462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 20 candidate_status { last_received { term: 19 index: 2924 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:43.188170 10462 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16901c5559a748a9aa770d73e254099c in current term 20: Already voted for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in this term.
I20260812 08:01:43.188328 11640 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183)
I20260812 08:01:43.188525  8006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 20 candidate_status { last_received { term: 19 index: 2924 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:43.188544 10729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 20 candidate_status { last_received { term: 19 index: 2924 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:43.188657 10729 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16901c5559a748a9aa770d73e254099c in current term 20: Already voted for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in this term.
I20260812 08:01:43.188642  8006 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16901c5559a748a9aa770d73e254099c in term 19.
I20260812 08:01:43.188627  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 20 candidate_status { last_received { term: 19 index: 2900 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:43.188791  8225 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df in current term 20: Already voted for candidate faafbc72cbd444ff90977a2cdb091b72 in this term.
I20260812 08:01:43.188954  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 20 candidate_status { last_received { term: 19 index: 2900 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:43.188989  8224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 20 candidate_status { last_received { term: 19 index: 2924 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:43.189101  7378 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df for term 20 because replica has last-logged OpId of term: 19 index: 2924, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 2900.
I20260812 08:01:43.189141  8224 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16901c5559a748a9aa770d73e254099c in current term 20: Already voted for candidate faafbc72cbd444ff90977a2cdb091b72 in this term.
I20260812 08:01:43.189148  7905 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 780d7e64a8214418a371f35351b113df; no voters: 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:43.189314 11640 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 19 FOLLOWER]: Advancing to term 20
I20260812 08:01:43.189855 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 20 candidate_status { last_received { term: 19 index: 2924 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:43.190001 10079 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16901c5559a748a9aa770d73e254099c in current term 20: Already voted for candidate faafbc72cbd444ff90977a2cdb091b72 in this term.
I20260812 08:01:43.190342  7284 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 780d7e64a8214418a371f35351b113df; no voters: 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:43.190513 11640 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260812 08:01:43.190538 11639 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 19 FOLLOWER]: Advancing to term 20
I20260812 08:01:43.191679 11639 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260812 08:01:44.413017 11641 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:01:44.413194 11641 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:44.414188 11641 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:44.414578 10462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 21 candidate_status { last_received { term: 19 index: 3010 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:44.414604 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 21 candidate_status { last_received { term: 19 index: 3010 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:44.414621  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 21 candidate_status { last_received { term: 19 index: 3010 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:44.414767 10079 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56b60424ed9f4a079689ad82a318d29a for term 21 because replica has last-logged OpId of term: 19 index: 3017, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 3010.
I20260812 08:01:44.414767 10462 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56b60424ed9f4a079689ad82a318d29a for term 21 because replica has last-logged OpId of term: 19 index: 3017, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 3010.
I20260812 08:01:44.414793  7378 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 20.
I20260812 08:01:44.414762  8224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 21 candidate_status { last_received { term: 19 index: 3010 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:44.414891  8224 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 20.
I20260812 08:01:44.415542  8006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 21 candidate_status { last_received { term: 19 index: 3010 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:44.415681  8006 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 20.
I20260812 08:01:44.415993 10623 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 3ae3f7a6e9f943e996a6dcea504878d0, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:44.416186 11641 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260812 08:01:44.416246 11641 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:01:44.416271 11641 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 20 FOLLOWER]: Advancing to term 21
I20260812 08:01:44.417296 11641 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:44.418092 11641 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 21 election: Requested vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:44.418150 10462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 21 candidate_status { last_received { term: 19 index: 3010 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0"
I20260812 08:01:44.418223 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 21 candidate_status { last_received { term: 19 index: 3010 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72"
I20260812 08:01:44.418223  8224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 21 candidate_status { last_received { term: 19 index: 3010 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
I20260812 08:01:44.418298 10462 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 20 FOLLOWER]: Advancing to term 21
I20260812 08:01:44.418370 10079 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 20 FOLLOWER]: Advancing to term 21
I20260812 08:01:44.418370  8224 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 20 FOLLOWER]: Advancing to term 21
I20260812 08:01:44.419162  8006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 21 candidate_status { last_received { term: 19 index: 3010 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:01:44.419296  8006 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 20 FOLLOWER]: Advancing to term 21
I20260812 08:01:44.419375  8224 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 21.
I20260812 08:01:44.419431 10079 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 56b60424ed9f4a079689ad82a318d29a for term 21 because replica has last-logged OpId of term: 19 index: 3017, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 3010.
I20260812 08:01:44.419443 10462 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 56b60424ed9f4a079689ad82a318d29a for term 21 because replica has last-logged OpId of term: 19 index: 3017, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 3010.
I20260812 08:01:44.419675  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 21 candidate_status { last_received { term: 19 index: 3010 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:01:44.419803  7378 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 20 FOLLOWER]: Advancing to term 21
I20260812 08:01:44.420382  8006 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 21.
I20260812 08:01:44.420841  7378 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 21.
I20260812 08:01:44.421167 10631 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 3ae3f7a6e9f943e996a6dcea504878d0, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:44.421366 11641 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 21 FOLLOWER]: Leader election won for term 21
I20260812 08:01:44.421538 11641 raft_consensus.cc:697] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 21 LEADER]: Becoming Leader. State: Replica: 56b60424ed9f4a079689ad82a318d29a, State: Running, Role: LEADER
I20260812 08:01:44.421697 11641 consensus_queue.cc:237] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3009, Committed index: 3009, Last appended: 19.3010, Last appended by leader: 3010, Current term: 21, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:44.423748  6748 catalog_manager.cc:5719] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a reported cstate change: term changed from 19 to 21, leader changed from 1664e32d3cd0473592085aa522379f22 (127.5.142.133) to 56b60424ed9f4a079689ad82a318d29a (127.5.142.130). New cstate: current_term: 21 leader_uuid: "56b60424ed9f4a079689ad82a318d29a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } health_report { overall_health: UNKNOWN } } }
W20260812 08:01:44.454557  9707 connection.cc:256] RPC call timeout handler was delayed by -2.2168523601721972s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 08:01:44.454841  9711 connection.cc:256] RPC call timeout handler was delayed by -2.4347368581220508s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 08:01:44.455003  9708 connection.cc:256] RPC call timeout handler was delayed by -2.3295344300568104s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 08:01:44.455300  9804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 20 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
W20260812 08:01:44.456986  9706 connection.cc:256] RPC call timeout handler was delayed by -2.5541145240422338s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 08:01:44.457509 10079 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 21 FOLLOWER]: Rejecting Update request from peer 1664e32d3cd0473592085aa522379f22 for earlier term 19. Current term is 21. Ops: []
I20260812 08:01:44.457650  9804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 20 candidate_status { last_received { term: 19 index: 2924 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:44.458091 11618 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 }, Status: INVALID_TERM, Last received: 19.3017, Next index: 3018, Last known committed idx: 3009, Time since last communication: 0.000s
I20260812 08:01:44.458252 11618 raft_consensus.cc:3055] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 19 LEADER]: Stepping down as leader of term 19
I20260812 08:01:44.458297 11618 raft_consensus.cc:740] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 1664e32d3cd0473592085aa522379f22, State: Running, Role: LEADER
I20260812 08:01:44.458442 11618 consensus_queue.cc:260] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3010, Committed index: 3010, Last appended: 19.3017, Last appended by leader: 3017, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:44.458585  9810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 20 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22"
I20260812 08:01:44.458702 10462 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 21 FOLLOWER]: Rejecting Update request from peer 1664e32d3cd0473592085aa522379f22 for earlier term 19. Current term is 21. Ops: []
I20260812 08:01:44.458748 11618 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 19 FOLLOWER]: Advancing to term 21
I20260812 08:01:44.458882  9801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 20 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22"
I20260812 08:01:44.459208  9809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 20 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:44.459532  9803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 21 candidate_status { last_received { term: 19 index: 3010 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:44.459774  9800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 2969 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:44.460021  9802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 21 candidate_status { last_received { term: 19 index: 3010 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22"
I20260812 08:01:44.460201  8224 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 21 FOLLOWER]: Rejecting Update request from peer 1664e32d3cd0473592085aa522379f22 for earlier term 19. Current term is 21. Ops: [19.2970-19.3017]
I20260812 08:01:44.460944  9808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 20 candidate_status { last_received { term: 19 index: 2900 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:44.461323  9806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 20 candidate_status { last_received { term: 19 index: 3010 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:44.461584  8006 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 21 FOLLOWER]: Rejecting Update request from peer 1664e32d3cd0473592085aa522379f22 for earlier term 19. Current term is 21. Ops: [19.2901-19.3017]
I20260812 08:01:44.461954  7378 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 21 FOLLOWER]: Rejecting Update request from peer 1664e32d3cd0473592085aa522379f22 for earlier term 19. Current term is 21. Ops: [19.2925-19.3017]
I20260812 08:01:44.462091  9810 raft_consensus.cc:2368] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate faafbc72cbd444ff90977a2cdb091b72 for earlier term 20. Current term is 21.
I20260812 08:01:44.463627  9966 leader_election.cc:400] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 20 election: Vote denied by peer 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105) with higher term. Message: Invalid argument: T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate faafbc72cbd444ff90977a2cdb091b72 for earlier term 20. Current term is 21.
I20260812 08:01:44.621340 11652 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:01:44.621582 11652 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:44.622435 11652 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:44.622920  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 22 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:44.622937 10729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 22 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:44.622920 10462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 22 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:44.623024  8006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 22 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:44.623077 10462 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 21.
I20260812 08:01:44.623080  7378 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 21.
I20260812 08:01:44.623169  8006 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 21.
I20260812 08:01:44.623570  9969 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, 780d7e64a8214418a371f35351b113df, faafbc72cbd444ff90977a2cdb091b72; no voters: 56b60424ed9f4a079689ad82a318d29a
I20260812 08:01:44.623684  8224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 22 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:44.623760 11652 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260812 08:01:44.623821 11652 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:01:44.623787  9784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 22 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:44.623826  8224 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 21.
I20260812 08:01:44.623908 11652 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 21 FOLLOWER]: Advancing to term 22
I20260812 08:01:44.623924  9784 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 21.
I20260812 08:01:44.624955 11652 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:44.625726 11652 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 22 election: Requested vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:44.625690 10462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 22 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0"
I20260812 08:01:44.625816 10462 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 21 FOLLOWER]: Advancing to term 22
I20260812 08:01:44.625782 10729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 22 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a"
I20260812 08:01:44.625918  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 22 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:01:44.626030  7378 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 21 FOLLOWER]: Advancing to term 22
I20260812 08:01:44.626137  9784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 22 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22"
I20260812 08:01:44.626250  9784 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 21 FOLLOWER]: Advancing to term 22
I20260812 08:01:44.626394  8224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 22 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
I20260812 08:01:44.626489  8224 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 21 FOLLOWER]: Advancing to term 22
I20260812 08:01:44.626845 10462 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 22.
I20260812 08:01:44.627064  7378 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 22.
I20260812 08:01:44.627326  9784 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 22.
I20260812 08:01:44.627345  8006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 22 candidate_status { last_received { term: 19 index: 3017 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:01:44.627454  8006 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 21 FOLLOWER]: Advancing to term 22
I20260812 08:01:44.627482  8224 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 22.
I20260812 08:01:44.627653  9966 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, faafbc72cbd444ff90977a2cdb091b72; no voters: 56b60424ed9f4a079689ad82a318d29a
I20260812 08:01:44.627861 11652 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 22 FOLLOWER]: Leader election won for term 22
I20260812 08:01:44.628041 11652 raft_consensus.cc:697] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 22 LEADER]: Becoming Leader. State: Replica: faafbc72cbd444ff90977a2cdb091b72, State: Running, Role: LEADER
I20260812 08:01:44.628208 11652 consensus_queue.cc:237] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3009, Committed index: 3009, Last appended: 19.3017, Last appended by leader: 3017, Current term: 22, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:44.628424  8006 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 22.
I20260812 08:01:44.630465  6748 catalog_manager.cc:5719] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 reported cstate change: term changed from 21 to 22, leader changed from 56b60424ed9f4a079689ad82a318d29a (127.5.142.130) to faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132). New cstate: current_term: 22 leader_uuid: "faafbc72cbd444ff90977a2cdb091b72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } health_report { overall_health: UNKNOWN } } }
I20260812 08:01:44.862140 10462 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 22 FOLLOWER]: Rejecting Update request from peer 56b60424ed9f4a079689ad82a318d29a for earlier term 21. Current term is 22. Ops: []
I20260812 08:01:44.862773 11641 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3011, Last known committed idx: 3009, Time since last communication: 0.000s
I20260812 08:01:44.862937 11641 consensus_queue.cc:1243] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Peer 3ae3f7a6e9f943e996a6dcea504878d0 log is divergent from this leader: its last log entry 19.3017 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3009
I20260812 08:01:44.863029 11642 raft_consensus.cc:3055] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 21 LEADER]: Stepping down as leader of term 21
I20260812 08:01:44.863121 11642 raft_consensus.cc:740] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 56b60424ed9f4a079689ad82a318d29a, State: Running, Role: LEADER
I20260812 08:01:44.863304 11642 consensus_queue.cc:260] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3009, Committed index: 3009, Last appended: 21.3011, Last appended by leader: 3011, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:44.863517 11642 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 21 FOLLOWER]: Advancing to term 22
I20260812 08:01:45.035988 10462 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 22 FOLLOWER]: Refusing update from remote peer faafbc72cbd444ff90977a2cdb091b72: Log matching property violated. Preceding OpId in replica: term: 19 index: 3017. Preceding OpId from leader: term: 22 index: 3018. (index mismatch)
I20260812 08:01:45.036525 11652 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3018, Last known committed idx: 3009, Time since last communication: 0.000s
I20260812 08:01:45.042698  8006 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 22 FOLLOWER]: Refusing update from remote peer faafbc72cbd444ff90977a2cdb091b72: Log matching property violated. Preceding OpId in replica: term: 19 index: 2900. Preceding OpId from leader: term: 22 index: 3018. (index mismatch)
I20260812 08:01:45.043169 11652 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3018, Last known committed idx: 2895, Time since last communication: 0.000s
I20260812 08:01:45.087404  8224 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 22 FOLLOWER]: Refusing update from remote peer faafbc72cbd444ff90977a2cdb091b72: Log matching property violated. Preceding OpId in replica: term: 19 index: 2969. Preceding OpId from leader: term: 22 index: 3018. (index mismatch)
I20260812 08:01:45.088030 11652 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3018, Last known committed idx: 2962, Time since last communication: 0.000s
I20260812 08:01:45.111647  7378 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 22 FOLLOWER]: Refusing update from remote peer faafbc72cbd444ff90977a2cdb091b72: Log matching property violated. Preceding OpId in replica: term: 19 index: 2924. Preceding OpId from leader: term: 22 index: 3018. (index mismatch)
I20260812 08:01:45.112119  9784 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 22 FOLLOWER]: Refusing update from remote peer faafbc72cbd444ff90977a2cdb091b72: Log matching property violated. Preceding OpId in replica: term: 19 index: 3017. Preceding OpId from leader: term: 22 index: 3018. (index mismatch)
I20260812 08:01:45.112849 10729 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 22 FOLLOWER]: Refusing update from remote peer faafbc72cbd444ff90977a2cdb091b72: Log matching property violated. Preceding OpId in replica: term: 21 index: 3011. Preceding OpId from leader: term: 22 index: 3018. (index mismatch)
I20260812 08:01:45.116607 11692 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3018, Last known committed idx: 3010, Time since last communication: 0.000s
I20260812 08:01:45.118615 11707 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48992 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2994, attempt_no=0}) took 3788 ms. Trace:
I20260812 08:01:45.116976 11699 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3018, Last known committed idx: 3009, Time since last communication: 0.002s
I20260812 08:01:45.119222 11699 consensus_queue.cc:1243] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Peer 56b60424ed9f4a079689ad82a318d29a log is divergent from this leader: its last log entry 21.3011 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3009
I20260812 08:01:45.118773 11707 rpcz_store.cc:276] 0812 08:01:41.330466 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:41.330577 (+   111us) service_pool.cc:224] Handling call
0812 08:01:45.118596 (+3788019us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:41.331297 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:41.331371 (+    74us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:41.331374 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:41.331375 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:41.335829 (+  4454us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:41.335835 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:41.335839 (+     4us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:41.335905 (+    66us) tablet.cc:701] Row locks acquired
0812 08:01:41.335906 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:41.335934 (+    28us) write_op.cc:270] Start()
0812 08:01:41.335953 (+    19us) write_op.cc:276] Timestamp: P: 1786521701335917 usec, L: 0
0812 08:01:41.335955 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:41.336877 (+   922us) log.cc:844] Serialized 696 byte log entry
0812 08:01:45.117577 (+3780700us) op_driver.cc:464] REPLICATION: finished
0812 08:01:45.117742 (+   165us) write_op.cc:301] APPLY: starting
0812 08:01:45.117758 (+    16us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:45.117795 (+    37us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:45.117810 (+    15us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:45.118050 (+   240us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:45.118083 (+    33us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:45.118088 (+     5us) write_op.cc:312] APPLY: finished
0812 08:01:45.118167 (+    79us) log.cc:844] Serialized 77 byte log entry
0812 08:01:45.118465 (+   298us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:45.118480 (+    15us) write_op.cc:454] Released schema lock
0812 08:01:45.118488 (+     8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":107,"num_ops":10,"prepare.queue_time_us":406,"prepare.run_cpu_time_us":564,"prepare.run_wall_time_us":6425,"raft.queue_time_us":51,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":305,"replication_time_us":3781597,"thread_start_us":170,"threads_started":2}]]}
I20260812 08:01:45.119481 11692 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3018, Last known committed idx: 2923, Time since last communication: 0.000s
I20260812 08:01:45.120199 11707 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48992 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2996, attempt_no=0}) took 3789 ms. Trace:
I20260812 08:01:45.120414 11707 rpcz_store.cc:276] 0812 08:01:41.331215 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:41.331300 (+    85us) service_pool.cc:224] Handling call
0812 08:01:45.120187 (+3788887us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:41.338231 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:41.338314 (+    83us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:41.338317 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:41.338318 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:41.338401 (+    83us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:41.338407 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:41.338410 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:41.338468 (+    58us) tablet.cc:701] Row locks acquired
0812 08:01:41.338469 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:41.338499 (+    30us) write_op.cc:270] Start()
0812 08:01:41.338519 (+    20us) write_op.cc:276] Timestamp: P: 1786521701338482 usec, L: 0
0812 08:01:41.338520 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:01:41.338854 (+   334us) log.cc:844] Serialized 686 byte log entry
0812 08:01:45.117814 (+3778960us) op_driver.cc:464] REPLICATION: finished
0812 08:01:45.119788 (+  1974us) write_op.cc:301] APPLY: starting
0812 08:01:45.119798 (+    10us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:45.119827 (+    29us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:45.119839 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:45.119993 (+   154us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:45.120020 (+    27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:45.120024 (+     4us) write_op.cc:312] APPLY: finished
0812 08:01:45.120081 (+    57us) log.cc:844] Serialized 77 byte log entry
0812 08:01:45.120114 (+    33us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:45.120127 (+    13us) write_op.cc:454] Released schema lock
0812 08:01:45.120132 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1944,"num_ops":10,"prepare.queue_time_us":6688,"prepare.run_cpu_time_us":424,"prepare.run_wall_time_us":1212,"replication_time_us":3779281,"thread_start_us":46,"threads_started":1}]]}
I20260812 08:01:45.121541 11707 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48992 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2997, attempt_no=0}) took 3788 ms. Trace:
I20260812 08:01:45.124662 11710 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48992 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2995, attempt_no=0}) took 3793 ms. Trace:
I20260812 08:01:45.124756 11710 rpcz_store.cc:276] 0812 08:01:41.330760 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:41.330807 (+    47us) service_pool.cc:224] Handling call
0812 08:01:45.124185 (+3793378us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:41.337756 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:41.337835 (+    79us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:41.337838 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:41.337839 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:41.337922 (+    83us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:41.337929 (+     7us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:41.337932 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:41.337994 (+    62us) tablet.cc:701] Row locks acquired
0812 08:01:41.337996 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:41.338025 (+    29us) write_op.cc:270] Start()
0812 08:01:41.338046 (+    21us) write_op.cc:276] Timestamp: P: 1786521701338007 usec, L: 0
0812 08:01:41.338047 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:01:41.338167 (+   120us) log.cc:844] Serialized 696 byte log entry
0812 08:01:45.117723 (+3779556us) op_driver.cc:464] REPLICATION: finished
0812 08:01:45.118128 (+   405us) write_op.cc:301] APPLY: starting
0812 08:01:45.122380 (+  4252us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:45.122411 (+    31us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:45.122424 (+    13us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:45.124001 (+  1577us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:45.124028 (+    27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:45.124032 (+     4us) write_op.cc:312] APPLY: finished
0812 08:01:45.124084 (+    52us) log.cc:844] Serialized 77 byte log entry
0812 08:01:45.124119 (+    35us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:45.124130 (+    11us) write_op.cc:454] Released schema lock
0812 08:01:45.124134 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":340,"num_ops":10,"prepare.queue_time_us":6691,"prepare.run_cpu_time_us":451,"prepare.run_wall_time_us":449,"replication_time_us":3779657,"thread_start_us":42,"threads_started":1}]]}
I20260812 08:01:45.125398 10729 raft_consensus.cc:1217] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 22 FOLLOWER]: Deduplicated request from leader. Original: 19.3009->[19.3010-22.3018]   Dedup: 19.3010->[19.3011-22.3018]
I20260812 08:01:45.125527 10729 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting all ops after (but not including) 3010
I20260812 08:01:45.125558 10729 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting uncommitted NO_OP operation due to leader change: 21.3011
I20260812 08:01:45.125584 10729 raft_consensus.cc:2889] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260812 08:01:45.125603 11710 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48992 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2999, attempt_no=0}) took 3789 ms. Trace:
I20260812 08:01:45.125653 11710 rpcz_store.cc:276] 0812 08:01:41.335780 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:41.336380 (+   600us) service_pool.cc:224] Handling call
0812 08:01:45.125592 (+3789212us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:41.339924 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:41.339981 (+    57us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:41.339984 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:41.339985 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:41.340045 (+    60us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:41.340049 (+     4us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:41.340051 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:41.340092 (+    41us) tablet.cc:701] Row locks acquired
0812 08:01:41.340093 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:41.340113 (+    20us) write_op.cc:270] Start()
0812 08:01:41.340127 (+    14us) write_op.cc:276] Timestamp: P: 1786521701340099 usec, L: 0
0812 08:01:41.340129 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:41.340218 (+    89us) log.cc:844] Serialized 696 byte log entry
0812 08:01:45.118055 (+3777837us) op_driver.cc:464] REPLICATION: finished
0812 08:01:45.125179 (+  7124us) write_op.cc:301] APPLY: starting
0812 08:01:45.125190 (+    11us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:45.125222 (+    32us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:45.125234 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:45.125383 (+   149us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:45.125410 (+    27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:45.125414 (+     4us) write_op.cc:312] APPLY: finished
0812 08:01:45.125473 (+    59us) log.cc:844] Serialized 77 byte log entry
0812 08:01:45.125512 (+    39us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:45.125526 (+    14us) write_op.cc:454] Released schema lock
0812 08:01:45.125531 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":7094,"mutex_wait_us":27,"num_ops":10,"prepare.queue_time_us":3290,"prepare.run_cpu_time_us":321,"prepare.run_wall_time_us":320,"replication_time_us":3777910}]]}
I20260812 08:01:45.126374 11710 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48992 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3001, attempt_no=0}) took 3789 ms. Trace:
I20260812 08:01:45.126428 11710 rpcz_store.cc:276] 0812 08:01:41.336553 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:41.336592 (+    39us) service_pool.cc:224] Handling call
0812 08:01:45.126363 (+3789771us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:41.340264 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:41.340315 (+    51us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:41.340317 (+     2us) write_op.cc:435] Acquired schema lock
0812 08:01:41.340318 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:41.340409 (+    91us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:41.340414 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:41.340418 (+     4us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:41.340473 (+    55us) tablet.cc:701] Row locks acquired
0812 08:01:41.340475 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:41.340503 (+    28us) write_op.cc:270] Start()
0812 08:01:41.340521 (+    18us) write_op.cc:276] Timestamp: P: 1786521701340486 usec, L: 0
0812 08:01:41.340523 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:41.340629 (+   106us) log.cc:844] Serialized 696 byte log entry
0812 08:01:45.118132 (+3777503us) op_driver.cc:464] REPLICATION: finished
0812 08:01:45.125979 (+  7847us) write_op.cc:301] APPLY: starting
0812 08:01:45.125989 (+    10us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:45.126017 (+    28us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:45.126027 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:45.126179 (+   152us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:45.126206 (+    27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:45.126211 (+     5us) write_op.cc:312] APPLY: finished
0812 08:01:45.126264 (+    53us) log.cc:844] Serialized 77 byte log entry
0812 08:01:45.126297 (+    33us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:45.126308 (+    11us) write_op.cc:454] Released schema lock
0812 08:01:45.126313 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":7807,"num_ops":10,"prepare.queue_time_us":3443,"prepare.run_cpu_time_us":394,"prepare.run_wall_time_us":393,"replication_time_us":3777596}]]}
W20260812 08:01:45.127385  9744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48992: Illegal state: replica 1664e32d3cd0473592085aa522379f22 is not leader of this config: current role FOLLOWER
I20260812 08:01:45.127923 11710 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48992 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3000, attempt_no=0}) took 3791 ms. Trace:
W20260812 08:01:45.127393  9743 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48992: Illegal state: replica 1664e32d3cd0473592085aa522379f22 is not leader of this config: current role FOLLOWER
W20260812 08:01:45.128212  9748 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48992: Illegal state: replica 1664e32d3cd0473592085aa522379f22 is not leader of this config: current role FOLLOWER
I20260812 08:01:45.128187 11710 rpcz_store.cc:276] 0812 08:01:41.336234 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:41.336661 (+   427us) service_pool.cc:224] Handling call
0812 08:01:45.127908 (+3791247us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:41.340687 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:41.340742 (+    55us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:41.340745 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:41.340746 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:41.340807 (+    61us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:41.340811 (+     4us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:41.340813 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:41.340853 (+    40us) tablet.cc:701] Row locks acquired
0812 08:01:41.340854 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:41.340874 (+    20us) write_op.cc:270] Start()
0812 08:01:41.340888 (+    14us) write_op.cc:276] Timestamp: P: 1786521701340859 usec, L: 0
0812 08:01:41.340911 (+    23us) op_driver.cc:348] REPLICATION: starting
0812 08:01:41.341009 (+    98us) log.cc:844] Serialized 696 byte log entry
0812 08:01:45.118163 (+3777154us) op_driver.cc:464] REPLICATION: finished
0812 08:01:45.126817 (+  8654us) write_op.cc:301] APPLY: starting
0812 08:01:45.126825 (+     8us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:45.126852 (+    27us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:45.126862 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:45.127054 (+   192us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:45.127079 (+    25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:45.127083 (+     4us) write_op.cc:312] APPLY: finished
0812 08:01:45.127449 (+   366us) log.cc:844] Serialized 77 byte log entry
0812 08:01:45.127640 (+   191us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:45.127654 (+    14us) write_op.cc:454] Released schema lock
0812 08:01:45.127660 (+     6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":8629,"num_ops":10,"prepare.queue_time_us":3820,"prepare.run_cpu_time_us":350,"prepare.run_wall_time_us":350,"replication_time_us":3777241,"spinlock_wait_cycles":376064}]]}
I20260812 08:01:45.129317 11708 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:48992 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=2998, attempt_no=0}) took 3793 ms. Trace:
I20260812 08:01:45.129387 11708 rpcz_store.cc:276] 0812 08:01:41.335709 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:41.336920 (+  1211us) service_pool.cc:224] Handling call
0812 08:01:45.129287 (+3792367us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:41.341058 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:41.341112 (+    54us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:41.341114 (+     2us) write_op.cc:435] Acquired schema lock
0812 08:01:41.341115 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:41.341186 (+    71us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:41.341190 (+     4us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:41.341192 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:41.341232 (+    40us) tablet.cc:701] Row locks acquired
0812 08:01:41.341232 (+     0us) write_op.cc:260] PREPARE: finished
0812 08:01:41.341253 (+    21us) write_op.cc:270] Start()
0812 08:01:41.341267 (+    14us) write_op.cc:276] Timestamp: P: 1786521701341238 usec, L: 0
0812 08:01:41.341268 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:01:41.341344 (+    76us) log.cc:844] Serialized 696 byte log entry
0812 08:01:45.118186 (+3776842us) op_driver.cc:464] REPLICATION: finished
0812 08:01:45.128837 (+ 10651us) write_op.cc:301] APPLY: starting
0812 08:01:45.128849 (+    12us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:45.128879 (+    30us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:45.128918 (+    39us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:45.129077 (+   159us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:45.129107 (+    30us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:45.129112 (+     5us) write_op.cc:312] APPLY: finished
0812 08:01:45.129179 (+    67us) log.cc:844] Serialized 77 byte log entry
0812 08:01:45.129215 (+    36us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:45.129228 (+    13us) write_op.cc:454] Released schema lock
0812 08:01:45.129232 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":10612,"num_ops":10,"prepare.queue_time_us":3928,"prepare.run_cpu_time_us":311,"prepare.run_wall_time_us":309,"replication_time_us":3776909}]]}
I20260812 08:01:45.121657 11707 rpcz_store.cc:276] 0812 08:01:41.332998 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:41.335495 (+  2497us) service_pool.cc:224] Handling call
0812 08:01:45.121530 (+3786035us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:41.339472 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:41.339540 (+    68us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:41.339544 (+     4us) write_op.cc:435] Acquired schema lock
0812 08:01:41.339546 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:41.339628 (+    82us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:41.339634 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:41.339637 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:41.339694 (+    57us) tablet.cc:701] Row locks acquired
0812 08:01:41.339696 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:41.339725 (+    29us) write_op.cc:270] Start()
0812 08:01:41.339744 (+    19us) write_op.cc:276] Timestamp: P: 1786521701339707 usec, L: 0
0812 08:01:41.339746 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:41.339858 (+   112us) log.cc:844] Serialized 686 byte log entry
0812 08:01:45.117909 (+3778051us) op_driver.cc:464] REPLICATION: finished
0812 08:01:45.121099 (+  3190us) write_op.cc:301] APPLY: starting
0812 08:01:45.121111 (+    12us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:45.121140 (+    29us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:45.121151 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:45.121330 (+   179us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:45.121360 (+    30us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:45.121364 (+     4us) write_op.cc:312] APPLY: finished
0812 08:01:45.121422 (+    58us) log.cc:844] Serialized 77 byte log entry
0812 08:01:45.121456 (+    34us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:45.121468 (+    12us) write_op.cc:454] Released schema lock
0812 08:01:45.121473 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3160,"num_ops":10,"prepare.queue_time_us":3704,"prepare.run_cpu_time_us":431,"prepare.run_wall_time_us":431,"replication_time_us":3778150,"thread_start_us":92,"threads_started":1}]]}
W20260812 08:01:45.131678  9748 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48992: Illegal state: replica 1664e32d3cd0473592085aa522379f22 is not leader of this config: current role FOLLOWER
W20260812 08:01:45.132310  9748 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48992: Illegal state: replica 1664e32d3cd0473592085aa522379f22 is not leader of this config: current role FOLLOWER
W20260812 08:01:45.132454  9744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48992: Illegal state: replica 1664e32d3cd0473592085aa522379f22 is not leader of this config: current role FOLLOWER
W20260812 08:01:45.132953  9743 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48992: Illegal state: replica 1664e32d3cd0473592085aa522379f22 is not leader of this config: current role FOLLOWER
W20260812 08:01:45.133625  9744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48992: Illegal state: replica 1664e32d3cd0473592085aa522379f22 is not leader of this config: current role FOLLOWER
W20260812 08:01:45.133812 10412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:45.135386 10416 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:45.136085 10412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:45.163690 10412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
I20260812 08:01:47.317173 11759 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:01:47.317337 11760 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:01:47.317353 11759 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:47.317481 11760 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:47.318323 11759 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:47.318657 11760 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:47.318747 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 23 candidate_status { last_received { term: 22 index: 3213 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:47.318750 10734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 23 candidate_status { last_received { term: 22 index: 3213 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:47.318806  9784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 23 candidate_status { last_received { term: 22 index: 3213 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:47.318881 10472 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 22.
I20260812 08:01:47.318909  9784 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 22.
I20260812 08:01:47.319120  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 23 candidate_status { last_received { term: 22 index: 3213 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:47.319145  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 23 candidate_status { last_received { term: 22 index: 3213 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:47.319438  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 23 candidate_status { last_received { term: 22 index: 3213 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:47.319695  8004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 23 candidate_status { last_received { term: 22 index: 3213 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:47.319844  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 23 candidate_status { last_received { term: 22 index: 3213 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:47.320040  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 23 candidate_status { last_received { term: 22 index: 3213 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:47.320375 10734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 23 candidate_status { last_received { term: 22 index: 3213 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:47.320441 10365 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0; no voters: 16901c5559a748a9aa770d73e254099c, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df
I20260812 08:01:47.320634 11759 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260812 08:01:47.320745  9707 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0; no voters: 16901c5559a748a9aa770d73e254099c, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df
I20260812 08:01:47.320933 11760 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260812 08:01:47.653216 11762 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 22 FOLLOWER]: Starting pre-election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:01:47.653376 11762 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:47.654206 11762 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183)
I20260812 08:01:47.654680 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 23 candidate_status { last_received { term: 22 index: 3093 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:47.654752  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 23 candidate_status { last_received { term: 22 index: 3093 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:47.654873 10472 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df for term 23 because replica has last-logged OpId of term: 22 index: 3213, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3093.
I20260812 08:01:47.654842  9784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 23 candidate_status { last_received { term: 22 index: 3093 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:47.654914  7381 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df for term 23 because replica has last-logged OpId of term: 22 index: 3123, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3093.
I20260812 08:01:47.654918 10734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 23 candidate_status { last_received { term: 22 index: 3093 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:47.654995  9784 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df for term 23 because replica has last-logged OpId of term: 22 index: 3213, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3093.
I20260812 08:01:47.655356  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 23 candidate_status { last_received { term: 22 index: 3093 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:47.655524  8225 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df for term 23 because replica has last-logged OpId of term: 22 index: 3165, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3093.
I20260812 08:01:47.655499 11766 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:01:47.655591 11766 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:47.655622  7902 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a
I20260812 08:01:47.655983 11762 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260812 08:01:47.656009 11768 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:01:47.656091 11768 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:47.656472 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 23 candidate_status { last_received { term: 22 index: 3123 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:47.656627 10472 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16901c5559a748a9aa770d73e254099c for term 23 because replica has last-logged OpId of term: 22 index: 3213, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3123.
I20260812 08:01:47.656631  9784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 23 candidate_status { last_received { term: 22 index: 3123 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:47.656659 10734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 23 candidate_status { last_received { term: 22 index: 3123 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:47.656777  9784 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16901c5559a748a9aa770d73e254099c for term 23 because replica has last-logged OpId of term: 22 index: 3213, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3123.
I20260812 08:01:47.656356 11766 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:47.657171  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 23 candidate_status { last_received { term: 22 index: 3123 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:47.657295  8008 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16901c5559a748a9aa770d73e254099c in term 22.
I20260812 08:01:47.657407  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 23 candidate_status { last_received { term: 22 index: 3123 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:47.657536  8225 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16901c5559a748a9aa770d73e254099c for term 23 because replica has last-logged OpId of term: 22 index: 3165, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3123.
I20260812 08:01:47.657882  9784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 3165 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:47.657871 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 3165 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:47.657908 10734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 3165 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:47.658008  9784 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6801d45ca6614156906e5d571ebcb8a6 for term 23 because replica has last-logged OpId of term: 22 index: 3213, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3165.
I20260812 08:01:47.658008 10472 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6801d45ca6614156906e5d571ebcb8a6 for term 23 because replica has last-logged OpId of term: 22 index: 3213, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3165.
I20260812 08:01:47.658103  7287 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6
I20260812 08:01:47.658215  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 3165 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:47.658311 11766 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260812 08:01:47.658349  8008 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6801d45ca6614156906e5d571ebcb8a6 in term 22.
I20260812 08:01:47.658607 11768 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:47.658674  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 3165 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:47.658797  7381 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6801d45ca6614156906e5d571ebcb8a6 in term 22.
I20260812 08:01:47.663630 11770 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:01:47.663730 11770 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:47.664425 11770 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:47.664538 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 23 candidate_status { last_received { term: 22 index: 3212 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:47.664697 10472 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56b60424ed9f4a079689ad82a318d29a for term 23 because replica has last-logged OpId of term: 22 index: 3213, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3212.
I20260812 08:01:47.664645  9784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 23 candidate_status { last_received { term: 22 index: 3212 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:47.664762  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 23 candidate_status { last_received { term: 22 index: 3212 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:47.664809  9784 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56b60424ed9f4a079689ad82a318d29a for term 23 because replica has last-logged OpId of term: 22 index: 3213, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3212.
I20260812 08:01:47.664873  8225 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 22.
I20260812 08:01:47.665311  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 23 candidate_status { last_received { term: 22 index: 3212 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:47.665446  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 23 candidate_status { last_received { term: 22 index: 3212 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:47.665447  7381 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 22.
I20260812 08:01:47.665572  8008 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 22.
I20260812 08:01:47.665864 10623 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0
I20260812 08:01:47.666036 11770 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260812 08:01:47.666098 11770 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 22 FOLLOWER]: Starting leader election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:01:47.666146 11770 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 22 FOLLOWER]: Advancing to term 23
I20260812 08:01:47.667054 11770 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:47.667693 11770 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 23 election: Requested vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:47.667800 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 23 candidate_status { last_received { term: 22 index: 3212 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0"
I20260812 08:01:47.667873  9784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 23 candidate_status { last_received { term: 22 index: 3212 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22"
I20260812 08:01:47.667910 10472 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 22 FOLLOWER]: Advancing to term 23
I20260812 08:01:47.667987  9784 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 22 FOLLOWER]: Advancing to term 23
I20260812 08:01:47.668203  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 23 candidate_status { last_received { term: 22 index: 3212 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:01:47.668309  8008 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 22 FOLLOWER]: Advancing to term 23
I20260812 08:01:47.668849 10472 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 56b60424ed9f4a079689ad82a318d29a for term 23 because replica has last-logged OpId of term: 22 index: 3213, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3212.
I20260812 08:01:47.669019  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 23 candidate_status { last_received { term: 22 index: 3212 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
I20260812 08:01:47.669134  8225 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 22 FOLLOWER]: Advancing to term 23
I20260812 08:01:47.669369  8008 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 23.
I20260812 08:01:47.670083  8225 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 23.
I20260812 08:01:47.670480  7381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 23 candidate_status { last_received { term: 22 index: 3212 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:01:47.670579  9784 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 56b60424ed9f4a079689ad82a318d29a for term 23 because replica has last-logged OpId of term: 22 index: 3213, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3212.
I20260812 08:01:47.670585  7381 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 22 FOLLOWER]: Advancing to term 23
I20260812 08:01:47.671365  7381 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 23.
I20260812 08:01:47.671648 10631 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0
I20260812 08:01:47.671828 11770 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 23 FOLLOWER]: Leader election won for term 23
I20260812 08:01:47.671998 11770 raft_consensus.cc:697] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 23 LEADER]: Becoming Leader. State: Replica: 56b60424ed9f4a079689ad82a318d29a, State: Running, Role: LEADER
I20260812 08:01:47.672156 11770 consensus_queue.cc:237] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3205, Committed index: 3205, Last appended: 22.3212, Last appended by leader: 3212, Current term: 23, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:47.674358  6745 catalog_manager.cc:5719] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a reported cstate change: term changed from 22 to 23, leader changed from faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132) to 56b60424ed9f4a079689ad82a318d29a (127.5.142.130). New cstate: current_term: 23 leader_uuid: "56b60424ed9f4a079689ad82a318d29a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } health_report { overall_health: UNKNOWN } } }
W20260812 08:01:48.001948  9966 connection.cc:256] RPC call timeout handler was delayed by -1.294066522968933s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 08:01:48.002218  9965 connection.cc:256] RPC call timeout handler was delayed by -1.6162574861664325s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 08:01:48.002355  9969 connection.cc:256] RPC call timeout handler was delayed by -1.5111125390976667s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 08:01:48.003772 10073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 23 candidate_status { last_received { term: 22 index: 3213 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:48.003894 11685 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59446 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3191, attempt_no=0}) took 2205 ms. Trace:
I20260812 08:01:48.004086 10072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 23 candidate_status { last_received { term: 22 index: 3213 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:48.003952 11685 rpcz_store.cc:276] 0812 08:01:45.798363 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:45.798398 (+    35us) service_pool.cc:224] Handling call
0812 08:01:48.003865 (+2205467us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:45.799551 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:45.799614 (+    63us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:45.799618 (+     4us) write_op.cc:435] Acquired schema lock
0812 08:01:45.799619 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:45.799699 (+    80us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:45.799704 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:45.799707 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:45.799767 (+    60us) tablet.cc:701] Row locks acquired
0812 08:01:45.799769 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:45.799799 (+    30us) write_op.cc:270] Start()
0812 08:01:45.799817 (+    18us) write_op.cc:276] Timestamp: P: 1786521705799784 usec, L: 0
0812 08:01:45.799819 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:45.799920 (+   101us) log.cc:844] Serialized 696 byte log entry
0812 08:01:48.003143 (+2203223us) op_driver.cc:464] REPLICATION: finished
0812 08:01:48.003245 (+   102us) write_op.cc:301] APPLY: starting
0812 08:01:48.003266 (+    21us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:48.003311 (+    45us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:48.003323 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:48.003510 (+   187us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:48.003544 (+    34us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:48.003548 (+     4us) write_op.cc:312] APPLY: finished
0812 08:01:48.003626 (+    78us) log.cc:844] Serialized 77 byte log entry
0812 08:01:48.003771 (+   145us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:48.003790 (+    19us) write_op.cc:454] Released schema lock
0812 08:01:48.003797 (+     7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":36,"num_ops":10,"prepare.queue_time_us":917,"prepare.run_cpu_time_us":432,"prepare.run_wall_time_us":630,"replication_time_us":2203291,"thread_start_us":76,"threads_started":1}]]}
I20260812 08:01:48.004307 10077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 3165 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:48.004493 10075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 23 candidate_status { last_received { term: 22 index: 3093 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:48.004725 11685 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59446 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3189, attempt_no=0}) took 2206 ms. Trace:
I20260812 08:01:48.004776 11685 rpcz_store.cc:276] 0812 08:01:45.798260 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:45.798593 (+   333us) service_pool.cc:224] Handling call
0812 08:01:48.004715 (+2206122us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:45.800206 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:45.800273 (+    67us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:45.800276 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:45.800277 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:45.800359 (+    82us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:45.800364 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:45.800367 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:45.800425 (+    58us) tablet.cc:701] Row locks acquired
0812 08:01:45.800426 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:45.800454 (+    28us) write_op.cc:270] Start()
0812 08:01:45.800471 (+    17us) write_op.cc:276] Timestamp: P: 1786521705800440 usec, L: 0
0812 08:01:45.800472 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:01:45.800574 (+   102us) log.cc:844] Serialized 696 byte log entry
0812 08:01:48.003204 (+2202630us) op_driver.cc:464] REPLICATION: finished
0812 08:01:48.004320 (+  1116us) write_op.cc:301] APPLY: starting
0812 08:01:48.004330 (+    10us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:48.004361 (+    31us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:48.004371 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:48.004536 (+   165us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:48.004561 (+    25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:48.004565 (+     4us) write_op.cc:312] APPLY: finished
0812 08:01:48.004617 (+    52us) log.cc:844] Serialized 77 byte log entry
0812 08:01:48.004649 (+    32us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:48.004663 (+    14us) write_op.cc:454] Released schema lock
0812 08:01:48.004667 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1093,"num_ops":10,"prepare.queue_time_us":1345,"prepare.run_cpu_time_us":442,"prepare.run_wall_time_us":747,"raft.queue_time_us":28,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":5,"replication_time_us":2202719,"thread_start_us":82,"threads_started":1}]]}
I20260812 08:01:48.005529 11685 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59446 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3192, attempt_no=0}) took 2206 ms. Trace:
I20260812 08:01:48.005580 11685 rpcz_store.cc:276] 0812 08:01:45.798715 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:45.798749 (+    34us) service_pool.cc:224] Handling call
0812 08:01:48.005521 (+2206772us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:45.800969 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:45.801046 (+    77us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:45.801049 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:45.801051 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:45.801130 (+    79us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:45.801135 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:45.801138 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:45.801194 (+    56us) tablet.cc:701] Row locks acquired
0812 08:01:45.801196 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:45.801223 (+    27us) write_op.cc:270] Start()
0812 08:01:45.801241 (+    18us) write_op.cc:276] Timestamp: P: 1786521705801209 usec, L: 0
0812 08:01:45.801242 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:01:45.801339 (+    97us) log.cc:844] Serialized 686 byte log entry
0812 08:01:48.003325 (+2201986us) op_driver.cc:464] REPLICATION: finished
0812 08:01:48.005152 (+  1827us) write_op.cc:301] APPLY: starting
0812 08:01:48.005162 (+    10us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:48.005190 (+    28us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:48.005201 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:48.005357 (+   156us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:48.005382 (+    25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:48.005386 (+     4us) write_op.cc:312] APPLY: finished
0812 08:01:48.005434 (+    48us) log.cc:844] Serialized 77 byte log entry
0812 08:01:48.005463 (+    29us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:48.005473 (+    10us) write_op.cc:454] Released schema lock
0812 08:01:48.005477 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1804,"num_ops":10,"prepare.queue_time_us":2003,"prepare.run_cpu_time_us":404,"prepare.run_wall_time_us":404,"replication_time_us":2202071,"thread_start_us":34,"threads_started":1}]]}
I20260812 08:01:48.005991  9784 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 23 FOLLOWER]: Rejecting Update request from peer faafbc72cbd444ff90977a2cdb091b72 for earlier term 22. Current term is 23. Ops: []
I20260812 08:01:48.006228 11685 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59446 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3190, attempt_no=0}) took 2207 ms. Trace:
I20260812 08:01:48.006276 11685 rpcz_store.cc:276] 0812 08:01:45.798314 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:45.798887 (+   573us) service_pool.cc:224] Handling call
0812 08:01:48.006220 (+2207333us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:45.801399 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:45.801466 (+    67us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:45.801469 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:45.801470 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:45.801549 (+    79us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:45.801554 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:45.801557 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:45.801615 (+    58us) tablet.cc:701] Row locks acquired
0812 08:01:45.801617 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:45.801643 (+    26us) write_op.cc:270] Start()
0812 08:01:45.801660 (+    17us) write_op.cc:276] Timestamp: P: 1786521705801630 usec, L: 0
0812 08:01:45.801662 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:45.801756 (+    94us) log.cc:844] Serialized 696 byte log entry
0812 08:01:48.003393 (+2201637us) op_driver.cc:464] REPLICATION: finished
0812 08:01:48.005850 (+  2457us) write_op.cc:301] APPLY: starting
0812 08:01:48.005859 (+     9us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:48.005885 (+    26us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:48.005895 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:48.006054 (+   159us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:48.006086 (+    32us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:48.006091 (+     5us) write_op.cc:312] APPLY: finished
0812 08:01:48.006136 (+    45us) log.cc:844] Serialized 77 byte log entry
0812 08:01:48.006165 (+    29us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:48.006174 (+     9us) write_op.cc:454] Released schema lock
0812 08:01:48.006179 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2436,"num_ops":10,"prepare.queue_time_us":2284,"prepare.run_cpu_time_us":401,"prepare.run_wall_time_us":401,"replication_time_us":2201721,"thread_start_us":47,"threads_started":1}]]}
I20260812 08:01:48.006933 11685 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59446 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3193, attempt_no=0}) took 2207 ms. Trace:
I20260812 08:01:48.006981 11685 rpcz_store.cc:276] 0812 08:01:45.799060 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:45.799093 (+    33us) service_pool.cc:224] Handling call
0812 08:01:48.006925 (+2207832us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:45.801829 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:45.801895 (+    66us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:45.801897 (+     2us) write_op.cc:435] Acquired schema lock
0812 08:01:45.801899 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:45.801979 (+    80us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:45.801985 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:45.801999 (+    14us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:45.802058 (+    59us) tablet.cc:701] Row locks acquired
0812 08:01:45.802059 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:45.802087 (+    28us) write_op.cc:270] Start()
0812 08:01:45.802104 (+    17us) write_op.cc:276] Timestamp: P: 1786521705802073 usec, L: 0
0812 08:01:45.802105 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:01:45.802231 (+   126us) log.cc:844] Serialized 686 byte log entry
0812 08:01:48.003473 (+2201242us) op_driver.cc:464] REPLICATION: finished
0812 08:01:48.006565 (+  3092us) write_op.cc:301] APPLY: starting
0812 08:01:48.006574 (+     9us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:48.006599 (+    25us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:48.006609 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:48.006767 (+   158us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:48.006790 (+    23us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:48.006794 (+     4us) write_op.cc:312] APPLY: finished
0812 08:01:48.006840 (+    46us) log.cc:844] Serialized 77 byte log entry
0812 08:01:48.006869 (+    29us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:48.006878 (+     9us) write_op.cc:454] Released schema lock
0812 08:01:48.006882 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3071,"num_ops":10,"prepare.queue_time_us":2501,"prepare.run_cpu_time_us":438,"prepare.run_wall_time_us":436,"replication_time_us":2201357}]]}
I20260812 08:01:48.007665 11685 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59446 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3195, attempt_no=0}) took 2203 ms. Trace:
I20260812 08:01:48.007711 11778 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59446 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3194, attempt_no=0}) took 2203 ms. Trace:
I20260812 08:01:48.007793 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 23 candidate_status { last_received { term: 22 index: 3212 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:48.008102  8008 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 23 FOLLOWER]: Rejecting Update request from peer faafbc72cbd444ff90977a2cdb091b72 for earlier term 22. Current term is 23. Ops: [22.3094-22.3213]
I20260812 08:01:48.007802 11778 rpcz_store.cc:276] 0812 08:01:45.804151 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:45.804230 (+    79us) service_pool.cc:224] Handling call
0812 08:01:48.007554 (+2203324us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:45.804551 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:45.804613 (+    62us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:45.804616 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:45.804618 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:45.804692 (+    74us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:45.804697 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:45.804699 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:45.804754 (+    55us) tablet.cc:701] Row locks acquired
0812 08:01:45.804756 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:45.804783 (+    27us) write_op.cc:270] Start()
0812 08:01:45.804800 (+    17us) write_op.cc:276] Timestamp: P: 1786521705804768 usec, L: 0
0812 08:01:45.804802 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:45.804940 (+   138us) log.cc:844] Serialized 696 byte log entry
0812 08:01:48.003494 (+2198554us) op_driver.cc:464] REPLICATION: finished
0812 08:01:48.007019 (+  3525us) write_op.cc:301] APPLY: starting
0812 08:01:48.007033 (+    14us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:48.007062 (+    29us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:48.007074 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:48.007361 (+   287us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:48.007386 (+    25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:48.007390 (+     4us) write_op.cc:312] APPLY: finished
0812 08:01:48.007444 (+    54us) log.cc:844] Serialized 77 byte log entry
0812 08:01:48.007486 (+    42us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:48.007499 (+    13us) write_op.cc:454] Released schema lock
0812 08:01:48.007504 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3493,"num_ops":10,"prepare.queue_time_us":67,"prepare.run_cpu_time_us":446,"prepare.run_wall_time_us":446,"replication_time_us":2198684}]]}
I20260812 08:01:48.008376 10734 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 23 LEADER]: Rejecting Update request from peer faafbc72cbd444ff90977a2cdb091b72 for earlier term 22. Current term is 23. Ops: [22.3213-22.3213]
I20260812 08:01:48.009254 10472 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 23 FOLLOWER]: Rejecting Update request from peer faafbc72cbd444ff90977a2cdb091b72 for earlier term 22. Current term is 23. Ops: []
I20260812 08:01:48.009665  8112 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:48.011266 10082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 23 candidate_status { last_received { term: 22 index: 3212 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72"
W20260812 08:01:48.002252  9968 connection.cc:256] RPC call timeout handler was delayed by -1.4029126402456313s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 08:01:48.012374 10082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 23 candidate_status { last_received { term: 22 index: 3123 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:48.012566 11768 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260812 08:01:48.008078 11733 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 }, Status: INVALID_TERM, Last received: 22.3213, Next index: 3214, Last known committed idx: 3205, Time since last communication: 0.006s
I20260812 08:01:48.007714 11685 rpcz_store.cc:276] 0812 08:01:45.804223 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:45.804493 (+   270us) service_pool.cc:224] Handling call
0812 08:01:48.007657 (+2203164us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:45.805020 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:45.805088 (+    68us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:45.805091 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:45.805093 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:45.805171 (+    78us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:45.805176 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:45.805179 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:45.805235 (+    56us) tablet.cc:701] Row locks acquired
0812 08:01:45.805237 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:45.805265 (+    28us) write_op.cc:270] Start()
0812 08:01:45.805284 (+    19us) write_op.cc:276] Timestamp: P: 1786521705805250 usec, L: 0
0812 08:01:45.805286 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:45.805388 (+   102us) log.cc:844] Serialized 696 byte log entry
0812 08:01:48.003513 (+2198125us) op_driver.cc:464] REPLICATION: finished
0812 08:01:48.007278 (+  3765us) write_op.cc:301] APPLY: starting
0812 08:01:48.007286 (+     8us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:48.007312 (+    26us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:48.007322 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:48.007495 (+   173us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:48.007519 (+    24us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:48.007523 (+     4us) write_op.cc:312] APPLY: finished
0812 08:01:48.007572 (+    49us) log.cc:844] Serialized 77 byte log entry
0812 08:01:48.007600 (+    28us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:48.007611 (+    11us) write_op.cc:454] Released schema lock
0812 08:01:48.007615 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3745,"num_ops":10,"prepare.queue_time_us":318,"prepare.run_cpu_time_us":418,"prepare.run_wall_time_us":417,"replication_time_us":2198220}]]}
I20260812 08:01:48.014406 11733 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 }, Status: INVALID_TERM, Last received: 22.3213, Next index: 3214, Last known committed idx: 3205, Time since last communication: 0.000s
I20260812 08:01:48.014673 11699 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 }, Status: INVALID_TERM, Last received: 22.3212, Next index: 3213, Last known committed idx: 3205, Time since last communication: 0.000s
I20260812 08:01:48.014781 11699 consensus_queue.cc:1243] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Peer 56b60424ed9f4a079689ad82a318d29a log is divergent from this leader: its last log entry 23.3213 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3205
I20260812 08:01:48.014961 11737 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 }, Status: INVALID_TERM, Last received: 22.3093, Next index: 3094, Last known committed idx: 3092, Time since last communication: 0.000s
I20260812 08:01:48.018786  8225 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 23 FOLLOWER]: Rejecting Update request from peer faafbc72cbd444ff90977a2cdb091b72 for earlier term 22. Current term is 23. Ops: [22.3166-22.3214]
I20260812 08:01:48.019665  7381 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 23 FOLLOWER]: Rejecting Update request from peer faafbc72cbd444ff90977a2cdb091b72 for earlier term 22. Current term is 23. Ops: [22.3124-22.3214]
I20260812 08:01:48.020368 11790 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 }, Status: INVALID_TERM, Last received: 22.3165, Next index: 3166, Last known committed idx: 3159, Time since last communication: 0.000s
I20260812 08:01:48.020812 11788 raft_consensus.cc:3055] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 22 LEADER]: Stepping down as leader of term 22
I20260812 08:01:48.020861 11788 raft_consensus.cc:740] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: faafbc72cbd444ff90977a2cdb091b72, State: Running, Role: LEADER
I20260812 08:01:48.021035 11788 consensus_queue.cc:260] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3212, Committed index: 3212, Last appended: 22.3217, Last appended by leader: 3217, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
W20260812 08:01:48.021363 11785 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.001s ago.
W20260812 08:01:48.022042 11785 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.001s ago.
W20260812 08:01:48.023033 11792 consensus_queue.cc:1175] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "16901c5559a748a9aa770d73e254099c" responder_term: 23 status { last_received { term: 22 index: 3123 } last_committed_idx: 3123 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer faafbc72cbd444ff90977a2cdb091b72 for earlier term 22. Current term is 23. Ops: [22.3124-22.3214]" } } last_received_current_leader { term: 0 index: 0 } }
I20260812 08:01:48.023576  9784 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 23 FOLLOWER]: Rejecting Update request from peer faafbc72cbd444ff90977a2cdb091b72 for earlier term 22. Current term is 23. Ops: [22.3214-22.3216]
I20260812 08:01:48.024118 11786 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 }, Status: INVALID_TERM, Last received: 22.3213, Next index: 3214, Last known committed idx: 3205, Time since last communication: 0.000s
I20260812 08:01:48.024549 10734 raft_consensus.cc:1217] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 23 LEADER]: Deduplicated request from leader. Original: 22.3205->[22.3206-22.3216]   Dedup: 22.3212->[22.3213-22.3216]
I20260812 08:01:48.024648 10734 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 23 LEADER]: Rejecting Update request from peer faafbc72cbd444ff90977a2cdb091b72 for earlier term 22. Current term is 23. Ops: [22.3206-22.3216]
I20260812 08:01:48.024747  8008 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 23 FOLLOWER]: Rejecting Update request from peer faafbc72cbd444ff90977a2cdb091b72 for earlier term 22. Current term is 23. Ops: [22.3094-22.3215]
W20260812 08:01:48.025332 11785 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.005s ago.
I20260812 08:01:48.025995 10472 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 23 FOLLOWER]: Rejecting Update request from peer faafbc72cbd444ff90977a2cdb091b72 for earlier term 22. Current term is 23. Ops: [22.3214-22.3216]
I20260812 08:01:48.026013 11788 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 22 FOLLOWER]: Advancing to term 23
W20260812 08:01:48.027731 10416 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:48.027760 10412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:48.031929 10412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:48.033095  9744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48992: Illegal state: replica 1664e32d3cd0473592085aa522379f22 is not leader of this config: current role FOLLOWER
W20260812 08:01:48.034071  9744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48992: Illegal state: replica 1664e32d3cd0473592085aa522379f22 is not leader of this config: current role FOLLOWER
W20260812 08:01:48.036283  9744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48992: Illegal state: replica 1664e32d3cd0473592085aa522379f22 is not leader of this config: current role FOLLOWER
I20260812 08:01:48.041109  7381 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 23 FOLLOWER]: Refusing update from remote peer 56b60424ed9f4a079689ad82a318d29a: Log matching property violated. Preceding OpId in replica: term: 22 index: 3123. Preceding OpId from leader: term: 23 index: 3214. (index mismatch)
I20260812 08:01:48.041297  9784 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 23 FOLLOWER]: Refusing update from remote peer 56b60424ed9f4a079689ad82a318d29a: Log matching property violated. Preceding OpId in replica: term: 22 index: 3213. Preceding OpId from leader: term: 23 index: 3214. (index mismatch)
I20260812 08:01:48.041638 10070 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 23 FOLLOWER]: Refusing update from remote peer 56b60424ed9f4a079689ad82a318d29a: Log matching property violated. Preceding OpId in replica: term: 22 index: 3217. Preceding OpId from leader: term: 23 index: 3214. (term mismatch)
I20260812 08:01:48.041720 10070 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting all ops after (but not including) 3213
I20260812 08:01:48.041745 10070 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 22.3214
I20260812 08:01:48.041721  8008 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 23 FOLLOWER]: Refusing update from remote peer 56b60424ed9f4a079689ad82a318d29a: Log matching property violated. Preceding OpId in replica: term: 22 index: 3093. Preceding OpId from leader: term: 23 index: 3214. (index mismatch)
I20260812 08:01:48.041843  8225 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 23 FOLLOWER]: Refusing update from remote peer 56b60424ed9f4a079689ad82a318d29a: Log matching property violated. Preceding OpId in replica: term: 22 index: 3165. Preceding OpId from leader: term: 23 index: 3214. (index mismatch)
W20260812 08:01:48.041863 10070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Aborted: Op aborted by new leader
I20260812 08:01:48.042142 10070 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 22.3215
I20260812 08:01:48.042152 11771 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3213, Last known committed idx: 3205, Time since last communication: 0.000s
I20260812 08:01:48.042301 11771 consensus_queue.cc:1243] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Peer 1664e32d3cd0473592085aa522379f22 log is divergent from this leader: its last log entry 22.3213 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3205
W20260812 08:01:48.042303 10070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Aborted: Op aborted by new leader
I20260812 08:01:48.042372 10472 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 23 FOLLOWER]: Refusing update from remote peer 56b60424ed9f4a079689ad82a318d29a: Log matching property violated. Preceding OpId in replica: term: 22 index: 3213. Preceding OpId from leader: term: 23 index: 3214. (index mismatch)
I20260812 08:01:48.042484 10070 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 22.3216
W20260812 08:01:48.042594 10070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Aborted: Op aborted by new leader
I20260812 08:01:48.042598 11770 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Connected to new peer: Peer: permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3213, Last known committed idx: 3123, Time since last communication: 0.000s
I20260812 08:01:48.042769 10070 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 22.3217
W20260812 08:01:48.042873 10070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Aborted: Op aborted by new leader
I20260812 08:01:48.042893 11771 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Connected to new peer: Peer: permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3213, Last known committed idx: 3092, Time since last communication: 0.000s
I20260812 08:01:48.043429  9784 raft_consensus.cc:1217] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 23 FOLLOWER]: Deduplicated request from leader. Original: 22.3205->[22.3206-23.3214]   Dedup: 22.3212->[23.3213-23.3214]
I20260812 08:01:48.043509  9784 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting all ops after (but not including) 3212
I20260812 08:01:48.043531  9784 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Aborting uncommitted WRITE_OP operation due to leader change: 22.3213
I20260812 08:01:48.043987 11770 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3213, Last known committed idx: 3159, Time since last communication: 0.000s
I20260812 08:01:48.044502 11806 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3213, Last known committed idx: 3205, Time since last communication: 0.000s
I20260812 08:01:48.044605 11806 consensus_queue.cc:1243] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Peer 3ae3f7a6e9f943e996a6dcea504878d0 log is divergent from this leader: its last log entry 22.3213 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3205
I20260812 08:01:48.046168 10472 raft_consensus.cc:1217] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 23 FOLLOWER]: Deduplicated request from leader. Original: 22.3205->[22.3206-23.3214]   Dedup: 22.3212->[23.3213-23.3214]
I20260812 08:01:48.046252 10472 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting all ops after (but not including) 3212
I20260812 08:01:48.046273 10472 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 22.3213
I20260812 08:01:48.047116 11771 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Connected to new peer: Peer: permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3213, Last known committed idx: 3212, Time since last communication: 0.000s
I20260812 08:01:48.047226 11771 consensus_queue.cc:1243] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [LEADER]: Peer faafbc72cbd444ff90977a2cdb091b72 log is divergent from this leader: its last log entry 22.3213 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3212
I20260812 08:01:48.049754 10070 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting all ops after (but not including) 3212
I20260812 08:01:48.049803 10070 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 22.3213
W20260812 08:01:48.049922 10070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Aborted: Op aborted by new leader
I20260812 08:01:48.050038 10070 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59446 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3196, attempt_no=0}) took 2235 ms. Trace:
I20260812 08:01:48.050087 10070 rpcz_store.cc:276] 0812 08:01:45.814345 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:45.814391 (+    46us) service_pool.cc:224] Handling call
0812 08:01:48.050029 (+2235638us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:45.814733 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:45.814810 (+    77us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:45.814813 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:45.814814 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:45.814905 (+    91us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:45.814911 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:45.814913 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:45.814973 (+    60us) tablet.cc:701] Row locks acquired
0812 08:01:45.814974 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:45.815003 (+    29us) write_op.cc:270] Start()
0812 08:01:45.815079 (+    76us) write_op.cc:276] Timestamp: P: 1786521705814987 usec, L: 0
0812 08:01:45.815081 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:45.815187 (+   106us) log.cc:844] Serialized 696 byte log entry
0812 08:01:48.049874 (+2234687us) op_driver.cc:464] REPLICATION: finished
0812 08:01:48.049880 (+     6us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 08:01:48.049884 (+     4us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:48.049911 (+    27us) write_op.cc:454] Released schema lock
0812 08:01:48.049917 (+     6us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":37,"prepare.run_cpu_time_us":529,"prepare.run_wall_time_us":1912,"raft.queue_time_us":52,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":220,"replication_time_us":2234750}]]}
I20260812 08:01:49.030548  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 10598
W20260812 08:01:49.045495  8305 meta_cache.cc:302] tablet f22cf4f6412248aa9e68c6d5d61153e8: replica 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363) has failed: Network error: recv got EOF from 127.5.142.130:38363 (error 108)
W20260812 08:01:49.051679  8116 connection.cc:466] server connection from 127.5.142.130:49441 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.130:49441 (request call id 252) could send its response
W20260812 08:01:49.053954  9707 connection.cc:466] server connection from 127.5.142.130:41241 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.130:41241 (request call id 259) could send its response
W20260812 08:01:49.056229  7907 connection.cc:466] server connection from 127.5.142.130:32865 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.130:32865 (request call id 78) could send its response
W20260812 08:01:49.056238  7286 connection.cc:466] server connection from 127.5.142.130:54727 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.130:54727 (request call id 100) could send its response
I20260812 08:01:50.217677 11902 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:50.217833 11902 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:50.218624 11902 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:50.219120 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 24 candidate_status { last_received { term: 23 index: 3403 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:50.219342 10472 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 23.
I20260812 08:01:50.219324  9783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 24 candidate_status { last_received { term: 23 index: 3403 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:50.219424 11903 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:50.219499 11903 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:50.219628  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 24 candidate_status { last_received { term: 23 index: 3403 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
W20260812 08:01:50.219686  9966 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260812 08:01:50.219965  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 24 candidate_status { last_received { term: 23 index: 3403 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:50.220479 10071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 24 candidate_status { last_received { term: 23 index: 3403 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:50.220602 10071 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 23.
I20260812 08:01:50.220623  9783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 24 candidate_status { last_received { term: 23 index: 3403 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
W20260812 08:01:50.220639  9966 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363): Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111)
I20260812 08:01:50.220717  9966 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 3ae3f7a6e9f943e996a6dcea504878d0, faafbc72cbd444ff90977a2cdb091b72; no voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 56b60424ed9f4a079689ad82a318d29a, 780d7e64a8214418a371f35351b113df
I20260812 08:01:50.221172  8224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 24 candidate_status { last_received { term: 23 index: 3403 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
W20260812 08:01:50.221253 10364 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260812 08:01:50.221238  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 24 candidate_status { last_received { term: 23 index: 3403 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:50.221318 11902 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260812 08:01:50.221184  8220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 24 candidate_status { last_received { term: 23 index: 3403 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:50.221539 11903 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:50.222079  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 24 candidate_status { last_received { term: 23 index: 3403 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
W20260812 08:01:50.222249 10364 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363): Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111)
I20260812 08:01:50.222325 10364 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 3ae3f7a6e9f943e996a6dcea504878d0, faafbc72cbd444ff90977a2cdb091b72; no voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6
I20260812 08:01:50.222473 11903 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260812 08:01:50.548627 11910 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:50.548812 11910 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:50.549806 11910 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183)
I20260812 08:01:50.550190 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 24 candidate_status { last_received { term: 23 index: 3282 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:50.550379 10472 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df for term 24 because replica has last-logged OpId of term: 23 index: 3403, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 3282.
I20260812 08:01:50.550431  9783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 24 candidate_status { last_received { term: 23 index: 3282 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:50.550522 10071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 24 candidate_status { last_received { term: 23 index: 3282 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:50.550689 10071 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df for term 24 because replica has last-logged OpId of term: 23 index: 3403, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 3282.
I20260812 08:01:50.550650  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 24 candidate_status { last_received { term: 23 index: 3282 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:50.550796  7378 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df for term 24 because replica has last-logged OpId of term: 23 index: 3312, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 3282.
I20260812 08:01:50.550909  8220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 24 candidate_status { last_received { term: 23 index: 3282 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:50.551082  8220 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df for term 24 because replica has last-logged OpId of term: 23 index: 3351, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 3282.
I20260812 08:01:50.551184 11915 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
W20260812 08:01:50.551301  7902 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111)
I20260812 08:01:50.551316 11915 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:50.551548  7902 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:50.551702 11910 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260812 08:01:50.552008 11917 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
W20260812 08:01:50.552089  7902 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363): Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111)
I20260812 08:01:50.552141 11917 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:50.552430 11915 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:50.552634 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 3351 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:50.552796 10472 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6801d45ca6614156906e5d571ebcb8a6 for term 24 because replica has last-logged OpId of term: 23 index: 3403, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 3351.
W20260812 08:01:50.552994  8116 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260812 08:01:50.553159  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 3351 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:50.553244  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 3351 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:50.553279  7378 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6801d45ca6614156906e5d571ebcb8a6 in term 23.
I20260812 08:01:50.553306 10073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 24 candidate_status { last_received { term: 23 index: 3312 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:50.553365  8008 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6801d45ca6614156906e5d571ebcb8a6 in term 23.
I20260812 08:01:50.553436 10073 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16901c5559a748a9aa770d73e254099c for term 24 because replica has last-logged OpId of term: 23 index: 3403, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 3312.
I20260812 08:01:50.553526 10462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 24 candidate_status { last_received { term: 23 index: 3312 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:50.553579 10071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 3351 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:50.553651 10462 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16901c5559a748a9aa770d73e254099c for term 24 because replica has last-logged OpId of term: 23 index: 3403, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 3312.
I20260812 08:01:50.553719 10071 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6801d45ca6614156906e5d571ebcb8a6 for term 24 because replica has last-logged OpId of term: 23 index: 3403, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 3351.
I20260812 08:01:50.554203  9784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 24 candidate_status { last_received { term: 23 index: 3312 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:50.554239  8220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 24 candidate_status { last_received { term: 23 index: 3312 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:50.554365  8220 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16901c5559a748a9aa770d73e254099c for term 24 because replica has last-logged OpId of term: 23 index: 3351, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 3312.
I20260812 08:01:50.554370  9784 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16901c5559a748a9aa770d73e254099c for term 24 because replica has last-logged OpId of term: 23 index: 3395, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 3312.
I20260812 08:01:50.554665 11917 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:50.554708  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 24 candidate_status { last_received { term: 23 index: 3312 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:50.554720 11923 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:50.554666  7286 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 16901c5559a748a9aa770d73e254099c; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:50.554805  8008 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16901c5559a748a9aa770d73e254099c in term 23.
I20260812 08:01:50.554905 11917 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260812 08:01:50.554854 11923 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:50.554896  9783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 3351 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:50.555061  9783 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6801d45ca6614156906e5d571ebcb8a6 for term 24 because replica has last-logged OpId of term: 23 index: 3395, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 3351.
W20260812 08:01:50.555431  7286 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111)
I20260812 08:01:50.555802 11923 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:50.555999 10071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 24 candidate_status { last_received { term: 23 index: 3395 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:50.556015  8220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 24 candidate_status { last_received { term: 23 index: 3395 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:50.556113  8220 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 23.
I20260812 08:01:50.556141 10071 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1664e32d3cd0473592085aa522379f22 for term 24 because replica has last-logged OpId of term: 23 index: 3403, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 3395.
W20260812 08:01:50.556345  8116 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363): Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111)
I20260812 08:01:50.556344 10462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 24 candidate_status { last_received { term: 23 index: 3395 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:50.556413  8116 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:50.556480 10462 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1664e32d3cd0473592085aa522379f22 for term 24 because replica has last-logged OpId of term: 23 index: 3403, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 3395.
I20260812 08:01:50.556481  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 24 candidate_status { last_received { term: 23 index: 3395 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:50.556593  7378 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 23.
I20260812 08:01:50.556830 11915 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260812 08:01:50.556994  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 24 candidate_status { last_received { term: 23 index: 3395 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
W20260812 08:01:50.557101  9707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111)
I20260812 08:01:50.557112  8008 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 23.
I20260812 08:01:50.557382  9706 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 3ae3f7a6e9f943e996a6dcea504878d0, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:50.557574 11923 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260812 08:01:50.557633 11923 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 56b60424ed9f4a079689ad82a318d29a)
I20260812 08:01:50.557674 11923 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 23 FOLLOWER]: Advancing to term 24
W20260812 08:01:50.557781  7286 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363): Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111)
W20260812 08:01:50.558001  9707 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363): Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111)
I20260812 08:01:50.558686 11923 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:50.559497  8220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 24 candidate_status { last_received { term: 23 index: 3395 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
I20260812 08:01:50.559496 10071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 24 candidate_status { last_received { term: 23 index: 3395 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72"
I20260812 08:01:50.559579 11923 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 24 election: Requested vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:50.559604 10071 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 23 FOLLOWER]: Advancing to term 24
I20260812 08:01:50.559602  8220 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 23 FOLLOWER]: Advancing to term 24
I20260812 08:01:50.559821  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 24 candidate_status { last_received { term: 23 index: 3395 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:01:50.559914  7378 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 23 FOLLOWER]: Advancing to term 24
I20260812 08:01:50.560459 10071 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 1664e32d3cd0473592085aa522379f22 for term 24 because replica has last-logged OpId of term: 23 index: 3403, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 3395.
I20260812 08:01:50.560707  8220 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 24.
I20260812 08:01:50.560871  7378 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 24.
I20260812 08:01:50.561002 10462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 24 candidate_status { last_received { term: 23 index: 3395 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0"
I20260812 08:01:50.561105 10462 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 23 FOLLOWER]: Advancing to term 24
W20260812 08:01:50.561836  9707 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363): Network error: Client connection negotiation failed: client connection to 127.5.142.130:38363: connect: Connection refused (error 111)
I20260812 08:01:50.561832  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 24 candidate_status { last_received { term: 23 index: 3395 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:01:50.561932  8008 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 23 FOLLOWER]: Advancing to term 24
I20260812 08:01:50.562110 10462 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 1664e32d3cd0473592085aa522379f22 for term 24 because replica has last-logged OpId of term: 23 index: 3403, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 3395.
I20260812 08:01:50.562834  8008 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 24.
I20260812 08:01:50.563105  9706 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:50.563270 11923 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 24 FOLLOWER]: Leader election won for term 24
I20260812 08:01:50.563443 11923 raft_consensus.cc:697] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 24 LEADER]: Becoming Leader. State: Replica: 1664e32d3cd0473592085aa522379f22, State: Running, Role: LEADER
I20260812 08:01:50.563617 11923 consensus_queue.cc:237] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3391, Committed index: 3391, Last appended: 23.3395, Last appended by leader: 3395, Current term: 24, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:50.565810  6746 catalog_manager.cc:5719] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 reported cstate change: term changed from 23 to 24, leader changed from 56b60424ed9f4a079689ad82a318d29a (127.5.142.130) to 1664e32d3cd0473592085aa522379f22 (127.5.142.133). New cstate: current_term: 24 leader_uuid: "1664e32d3cd0473592085aa522379f22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } health_report { overall_health: UNKNOWN } } }
I20260812 08:01:50.603854  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.130:38363
--local_ip_for_outbound_sockets=127.5.142.130
--tserver_master_addrs=127.5.142.190:44379
--webserver_port=38317
--webserver_interface=127.5.142.130
--builtin_ntp_servers=127.5.142.148:43055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
W20260812 08:01:50.716701 11937 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:50.716984 11937 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:50.717043 11937 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:01:50.720415 11937 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:50.720559 11937 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.130
I20260812 08:01:50.724574 11937 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:43055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.130:38363
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.5.142.130
--webserver_port=38317
--tserver_master_addrs=127.5.142.190:44379
--never_fsync=true
--heap_profile_path=/tmp/kudu.11937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.130

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:50.725737 11937 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:50.726895 11937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:50.733575 11951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:50.733582 11946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:50.733595 11945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:50.733687 11937 server_base.cc:1061] running on GCE node
I20260812 08:01:50.734127 11937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:50.734642 11937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:50.735836 11937 hybrid_clock.cc:648] HybridClock initialized: now 1786521710735817 us; error 34 us; skew 500 ppm
I20260812 08:01:50.737802 11937 webserver.cc:533] Webserver started at http://127.5.142.130:38317/ using document root <none> and password file <none>
I20260812 08:01:50.738380 11937 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:50.738443 11937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:50.741815 11937 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.003s	sys 0.001s
I20260812 08:01:50.743714 11960 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:50.744678 11937 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:01:50.744786 11937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "56b60424ed9f4a079689ad82a318d29a"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-ncp9"
I20260812 08:01:50.745217 11937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:50.756976 11937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:50.757622 11937 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:50.757789 11937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:50.758319 11937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:50.759568 11971 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:01:50.763025 11937 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:01:50.763082 11937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 08:01:50.763150 11937 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:01:50.764583 11937 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:01:50.764628 11937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
I20260812 08:01:50.764963 11971 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Bootstrap starting.
I20260812 08:01:50.774247 11937 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.130:38363
I20260812 08:01:50.774312 12138 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.130:38363 every 8 connection(s)
I20260812 08:01:50.775199 11937 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:01:50.782464  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 11937
I20260812 08:01:50.787120 12140 heartbeater.cc:344] Connected to a master server at 127.5.142.190:44379
I20260812 08:01:50.787391 12140 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:50.788000 12140 heartbeater.cc:507] Master 127.5.142.190:44379 requested a full tablet report, sending...
I20260812 08:01:50.790032  6746 ts_manager.cc:194] Re-registered known tserver with Master: 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363)
I20260812 08:01:50.790884  6746 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.130:56489
I20260812 08:01:50.959849 11971 log.cc:826] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:50.966279  7378 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 24 FOLLOWER]: Refusing update from remote peer 1664e32d3cd0473592085aa522379f22: Log matching property violated. Preceding OpId in replica: term: 23 index: 3312. Preceding OpId from leader: term: 24 index: 3396. (index mismatch)
I20260812 08:01:50.966817 11923 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3396, Last known committed idx: 3307, Time since last communication: 0.000s
I20260812 08:01:50.989590  8220 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 24 FOLLOWER]: Refusing update from remote peer 1664e32d3cd0473592085aa522379f22: Log matching property violated. Preceding OpId in replica: term: 23 index: 3351. Preceding OpId from leader: term: 24 index: 3396. (index mismatch)
I20260812 08:01:50.990161 11923 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3396, Last known committed idx: 3347, Time since last communication: 0.000s
I20260812 08:01:51.135309  8008 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 24 FOLLOWER]: Refusing update from remote peer 1664e32d3cd0473592085aa522379f22: Log matching property violated. Preceding OpId in replica: term: 23 index: 3282. Preceding OpId from leader: term: 24 index: 3396. (index mismatch)
I20260812 08:01:51.135972 11923 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3396, Last known committed idx: 3281, Time since last communication: 0.000s
I20260812 08:01:51.148036 10071 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 24 FOLLOWER]: Refusing update from remote peer 1664e32d3cd0473592085aa522379f22: Log matching property violated. Preceding OpId in replica: term: 23 index: 3403. Preceding OpId from leader: term: 24 index: 3396. (term mismatch)
I20260812 08:01:51.148161 10071 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting all ops after (but not including) 3395
I20260812 08:01:51.148195 10071 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 23.3396
I20260812 08:01:51.148424 10071 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 23.3397
I20260812 08:01:51.148619 10071 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 23.3398
I20260812 08:01:51.148783 10071 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 23.3399
I20260812 08:01:51.148969 10071 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 23.3400
I20260812 08:01:51.149123 10071 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 23.3401
I20260812 08:01:51.149312 10071 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 23.3402
I20260812 08:01:51.149490 10071 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72: Aborting uncommitted WRITE_OP operation due to leader change: 23.3403
I20260812 08:01:51.150226 11923 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3396, Last known committed idx: 3395, Time since last communication: 0.000s
I20260812 08:01:51.158100 10462 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 24 FOLLOWER]: Refusing update from remote peer 1664e32d3cd0473592085aa522379f22: Log matching property violated. Preceding OpId in replica: term: 23 index: 3403. Preceding OpId from leader: term: 24 index: 3396. (term mismatch)
I20260812 08:01:51.158197 10462 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting all ops after (but not including) 3395
I20260812 08:01:51.158224 10462 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 23.3396
I20260812 08:01:51.158497 10462 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3378, attempt_no=1}) took 2100 ms. Trace:
I20260812 08:01:51.158564 10462 rpcz_store.cc:276] 0812 08:01:49.058478 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:49.058509 (+    31us) service_pool.cc:224] Handling call
0812 08:01:51.158484 (+2099975us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 08:01:51.158818 10462 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 23.3397
I20260812 08:01:51.158959 10462 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3379, attempt_no=1}) took 2100 ms. Trace:
I20260812 08:01:51.158996 10462 rpcz_store.cc:276] 0812 08:01:49.058302 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:49.058346 (+    44us) service_pool.cc:224] Handling call
0812 08:01:51.158953 (+2100607us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 08:01:51.159122 10462 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 23.3398
I20260812 08:01:51.159266 10462 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3380, attempt_no=1}) took 2101 ms. Trace:
I20260812 08:01:51.159312 10462 rpcz_store.cc:276] 0812 08:01:49.058219 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:49.058266 (+    47us) service_pool.cc:224] Handling call
0812 08:01:51.159257 (+2100991us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 08:01:51.159442 10462 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 23.3399
I20260812 08:01:51.159574 10462 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3381, attempt_no=1}) took 2100 ms. Trace:
I20260812 08:01:51.159612 10462 rpcz_store.cc:276] 0812 08:01:49.058622 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:49.058653 (+    31us) service_pool.cc:224] Handling call
0812 08:01:51.159567 (+2100914us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 08:01:51.159745 10462 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 23.3400
I20260812 08:01:51.159873 10462 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3382, attempt_no=1}) took 2101 ms. Trace:
I20260812 08:01:51.159914 10462 rpcz_store.cc:276] 0812 08:01:49.058556 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:49.058588 (+    32us) service_pool.cc:224] Handling call
0812 08:01:51.159865 (+2101277us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 08:01:51.160037 10462 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 23.3401
I20260812 08:01:51.160171 10462 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3384, attempt_no=1}) took 2101 ms. Trace:
I20260812 08:01:51.160215 10462 rpcz_store.cc:276] 0812 08:01:49.058363 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:49.058389 (+    26us) service_pool.cc:224] Handling call
0812 08:01:51.160163 (+2101774us) inbound_call.cc:177] Queueing failure response
Metrics: {"spinlock_wait_cycles":3456}
I20260812 08:01:51.160367 10462 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 23.3402
I20260812 08:01:51.160495 10462 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3383, attempt_no=1}) took 2102 ms. Trace:
I20260812 08:01:51.160537 10462 rpcz_store.cc:276] 0812 08:01:49.058421 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:49.058833 (+   412us) service_pool.cc:224] Handling call
0812 08:01:51.160488 (+2101655us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 08:01:51.160696 10462 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0: Aborting uncommitted WRITE_OP operation due to leader change: 23.3403
I20260812 08:01:51.160833 10462 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3385, attempt_no=1}) took 2102 ms. Trace:
W20260812 08:01:51.173246  9707 consensus_peers.cc:597] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 -> Peer 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363): Couldn't send request to peer 56b60424ed9f4a079689ad82a318d29a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260812 08:01:51.174381 10411 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:51.174965 10416 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:51.175761 10416 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:51.176671 10415 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:51.177268 10411 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:51.177780 10416 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
I20260812 08:01:51.160874 10462 rpcz_store.cc:276] 0812 08:01:49.058681 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:49.058706 (+    25us) service_pool.cc:224] Handling call
0812 08:01:51.160825 (+2102119us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 08:01:51.179262 12191 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3396, Last known committed idx: 3395, Time since last communication: 0.000s
W20260812 08:01:51.180989 10009 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Illegal state: replica faafbc72cbd444ff90977a2cdb091b72 is not leader of this config: current role FOLLOWER
W20260812 08:01:51.181803 10412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:51.183792 10006 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Illegal state: replica faafbc72cbd444ff90977a2cdb091b72 is not leader of this config: current role FOLLOWER
W20260812 08:01:51.184171 10009 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Illegal state: replica faafbc72cbd444ff90977a2cdb091b72 is not leader of this config: current role FOLLOWER
W20260812 08:01:51.185144 10005 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Illegal state: replica faafbc72cbd444ff90977a2cdb091b72 is not leader of this config: current role FOLLOWER
W20260812 08:01:51.185333 10009 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Illegal state: replica faafbc72cbd444ff90977a2cdb091b72 is not leader of this config: current role FOLLOWER
W20260812 08:01:51.187032 10006 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Illegal state: replica faafbc72cbd444ff90977a2cdb091b72 is not leader of this config: current role FOLLOWER
W20260812 08:01:51.187917 10006 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Illegal state: replica faafbc72cbd444ff90977a2cdb091b72 is not leader of this config: current role FOLLOWER
I20260812 08:01:51.792665 12140 heartbeater.cc:499] Master 127.5.142.190:44379 was elected leader, sending a full tablet report...
I20260812 08:01:52.320261  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 9688
W20260812 08:01:52.336879  8305 meta_cache.cc:302] tablet f22cf4f6412248aa9e68c6d5d61153e8: replica 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105) has failed: Network error: recv got EOF from 127.5.142.133:33105 (error 108)
W20260812 08:01:52.343353  8112 connection.cc:466] server connection from 127.5.142.133:55353 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.133:55353 (request call id 190) could send its response
W20260812 08:01:52.347229 11961 connection.cc:466] server connection from 127.5.142.133:47407 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.133:47407 (request call id 2) could send its response
W20260812 08:01:52.349436  7907 connection.cc:466] server connection from 127.5.142.133:36115 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.133:36115 (request call id 102) could send its response
W20260812 08:01:52.352574  7288 connection.cc:466] server connection from 127.5.142.133:49707 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.142.133:49707 (request call id 174) could send its response
I20260812 08:01:53.392470 12234 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1664e32d3cd0473592085aa522379f22)
I20260812 08:01:53.392647 12234 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:53.393529 12234 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:53.393852 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 25 candidate_status { last_received { term: 24 index: 3557 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:53.394030 10472 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 24.
I20260812 08:01:53.394232 12235 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1664e32d3cd0473592085aa522379f22)
I20260812 08:01:53.394227  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 25 candidate_status { last_received { term: 24 index: 3557 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:53.394341 12235 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:53.394958  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 25 candidate_status { last_received { term: 24 index: 3557 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:53.395009 12235 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:53.395864 10071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 25 candidate_status { last_received { term: 24 index: 3557 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:53.395957  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 25 candidate_status { last_received { term: 24 index: 3557 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:53.396014 10071 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 24.
I20260812 08:01:53.396028  8220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 25 candidate_status { last_received { term: 24 index: 3557 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:53.396566  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 25 candidate_status { last_received { term: 24 index: 3557 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:53.396775  8220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 25 candidate_status { last_received { term: 24 index: 3557 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
W20260812 08:01:53.398792  9966 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105): Network error: Client connection negotiation failed: client connection to 127.5.142.133:33105: connect: Connection refused (error 111)
I20260812 08:01:53.398864  9966 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 3ae3f7a6e9f943e996a6dcea504878d0, faafbc72cbd444ff90977a2cdb091b72; no voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df
I20260812 08:01:53.399013 12234 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260812 08:01:53.399819 10364 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105): Network error: Client connection negotiation failed: client connection to 127.5.142.133:33105: connect: Connection refused (error 111)
I20260812 08:01:53.399900 10364 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 3ae3f7a6e9f943e996a6dcea504878d0, faafbc72cbd444ff90977a2cdb091b72; no voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df
I20260812 08:01:53.400127 12235 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260812 08:01:53.401770 12074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 25 candidate_status { last_received { term: 24 index: 3557 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:53.401655 12075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 25 candidate_status { last_received { term: 24 index: 3557 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
W20260812 08:01:53.403189 10364 leader_election.cc:343] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 25 pre-election: Tablet error from VoteRequest() call to peer 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:01:53.839552 12250 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1664e32d3cd0473592085aa522379f22)
I20260812 08:01:53.839720 12250 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:53.840874 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 25 candidate_status { last_received { term: 24 index: 3488 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:53.841110 10472 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df for term 25 because replica has last-logged OpId of term: 24 index: 3557, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 3488.
I20260812 08:01:53.841152 10071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 25 candidate_status { last_received { term: 24 index: 3488 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:53.841341 10071 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df for term 25 because replica has last-logged OpId of term: 24 index: 3557, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 3488.
I20260812 08:01:53.841352 12250 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183)
I20260812 08:01:53.842278  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 25 candidate_status { last_received { term: 24 index: 3488 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:53.842700  8220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 25 candidate_status { last_received { term: 24 index: 3488 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:53.842810 12255 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1664e32d3cd0473592085aa522379f22)
I20260812 08:01:53.842916 12255 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
W20260812 08:01:53.843134  7902 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105): Network error: Client connection negotiation failed: client connection to 127.5.142.133:33105: connect: Connection refused (error 111)
I20260812 08:01:53.843209  7902 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:53.843380 12250 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260812 08:01:53.843418  8220 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 780d7e64a8214418a371f35351b113df for term 25 because replica has last-logged OpId of term: 24 index: 3550, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 3488.
I20260812 08:01:53.845100 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 3550 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:53.845256 10472 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6801d45ca6614156906e5d571ebcb8a6 for term 25 because replica has last-logged OpId of term: 24 index: 3557, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 3550.
I20260812 08:01:53.845762 10071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 3550 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:53.845920 10071 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6801d45ca6614156906e5d571ebcb8a6 for term 25 because replica has last-logged OpId of term: 24 index: 3557, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 3550.
I20260812 08:01:53.846769 12255 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:53.847121  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 3550 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:53.847246  8008 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6801d45ca6614156906e5d571ebcb8a6 in term 24.
I20260812 08:01:53.847476  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 3550 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:53.847620 12074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 25 candidate_status { last_received { term: 24 index: 3488 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
W20260812 08:01:53.847965  7902 leader_election.cc:343] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 25 pre-election: Tablet error from VoteRequest() call to peer 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363): Illegal state: must be running to vote when last-logged opid is not known
W20260812 08:01:53.848067  8116 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105): Network error: Client connection negotiation failed: client connection to 127.5.142.133:33105: connect: Connection refused (error 111)
I20260812 08:01:53.848137  8116 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 3ae3f7a6e9f943e996a6dcea504878d0, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:53.848332 12255 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260812 08:01:53.850095 12262 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1664e32d3cd0473592085aa522379f22)
I20260812 08:01:53.850200 12262 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:53.851239 12074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 3550 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:53.851691 12262 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:53.851919 10071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 25 candidate_status { last_received { term: 24 index: 3514 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:53.851971 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 25 candidate_status { last_received { term: 24 index: 3514 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:53.852089 10071 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16901c5559a748a9aa770d73e254099c for term 25 because replica has last-logged OpId of term: 24 index: 3557, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 3514.
I20260812 08:01:53.852109 10472 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16901c5559a748a9aa770d73e254099c for term 25 because replica has last-logged OpId of term: 24 index: 3557, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 3514.
I20260812 08:01:53.852178  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 25 candidate_status { last_received { term: 24 index: 3514 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:53.852325  8008 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16901c5559a748a9aa770d73e254099c in term 24.
I20260812 08:01:53.852553  8220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 25 candidate_status { last_received { term: 24 index: 3514 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:53.852705  8220 raft_consensus.cc:2410] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16901c5559a748a9aa770d73e254099c for term 25 because replica has last-logged OpId of term: 24 index: 3550, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 3514.
W20260812 08:01:53.853170  8116 leader_election.cc:343] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 25 pre-election: Tablet error from VoteRequest() call to peer 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363): Illegal state: must be running to vote when last-logged opid is not known
W20260812 08:01:53.854532  7286 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105): Network error: Client connection negotiation failed: client connection to 127.5.142.133:33105: connect: Connection refused (error 111)
I20260812 08:01:53.854610  7286 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 16901c5559a748a9aa770d73e254099c, 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:53.854748 12262 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260812 08:01:53.856566 12074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "16901c5559a748a9aa770d73e254099c" candidate_term: 25 candidate_status { last_received { term: 24 index: 3514 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
W20260812 08:01:53.856925  7286 leader_election.cc:343] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [CANDIDATE]: Term 25 pre-election: Tablet error from VoteRequest() call to peer 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:01:53.929950 11971 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Bootstrap replayed 1/1 log segments. Stats: ops{read=3422 overwritten=19 applied=3395 ignored=0} inserts{seen=33780 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 8 replicates
I20260812 08:01:53.930598 11971 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Bootstrap complete.
I20260812 08:01:53.931814 11971 ts_tablet_manager.cc:1403] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Time spent bootstrapping tablet: real 3.167s	user 2.574s	sys 0.112s
I20260812 08:01:53.933470 11971 raft_consensus.cc:359] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 23 FOLLOWER]: Replica starting. Triggering 8 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:53.935438 11971 raft_consensus.cc:740] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 23 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56b60424ed9f4a079689ad82a318d29a, State: Initialized, Role: FOLLOWER
I20260812 08:01:53.935897 11971 consensus_queue.cc:260] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3395, Last appended: 23.3403, Last appended by leader: 3403, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:53.936697 11971 ts_tablet_manager.cc:1434] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Time spent starting tablet: real 0.005s	user 0.007s	sys 0.000s
I20260812 08:01:54.906719 12278 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1664e32d3cd0473592085aa522379f22)
I20260812 08:01:54.906899 12278 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:54.908062 12278 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:54.908021 10071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 25 candidate_status { last_received { term: 24 index: 3557 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:54.908046  8220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 25 candidate_status { last_received { term: 24 index: 3557 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:54.908195 10071 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 24.
I20260812 08:01:54.908200  8220 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 24.
I20260812 08:01:54.908324 12074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 25 candidate_status { last_received { term: 24 index: 3557 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:54.908571 12074 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 23.
I20260812 08:01:54.908556  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 25 candidate_status { last_received { term: 24 index: 3557 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:54.908708  7378 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 24.
I20260812 08:01:54.908984 10364 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72; no voters: 
I20260812 08:01:54.909178 12278 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260812 08:01:54.909232 12278 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 1664e32d3cd0473592085aa522379f22)
I20260812 08:01:54.909276 12278 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 24 FOLLOWER]: Advancing to term 25
I20260812 08:01:54.909387  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 25 candidate_status { last_received { term: 24 index: 3557 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:54.909529  8008 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 24.
W20260812 08:01:54.909720 10364 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105): Network error: Client connection negotiation failed: client connection to 127.5.142.133:33105: connect: Connection refused (error 111)
I20260812 08:01:54.910451 12278 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:54.911017 12278 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 25 election: Requested vote from peers faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:54.911381  8220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 25 candidate_status { last_received { term: 24 index: 3557 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
I20260812 08:01:54.911478 12074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 25 candidate_status { last_received { term: 24 index: 3557 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a"
I20260812 08:01:54.911540  8220 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 24 FOLLOWER]: Advancing to term 25
I20260812 08:01:54.911598 12074 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 23 FOLLOWER]: Advancing to term 25
I20260812 08:01:54.911625  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 25 candidate_status { last_received { term: 24 index: 3557 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:01:54.911731  8008 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 24 FOLLOWER]: Advancing to term 25
I20260812 08:01:54.912518  8008 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 25.
I20260812 08:01:54.912676  8220 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 25.
I20260812 08:01:54.913045  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 25 candidate_status { last_received { term: 24 index: 3557 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:01:54.913071 10071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 25 candidate_status { last_received { term: 24 index: 3557 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72"
I20260812 08:01:54.913192  7378 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 24 FOLLOWER]: Advancing to term 25
I20260812 08:01:54.913203 10071 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 24 FOLLOWER]: Advancing to term 25
W20260812 08:01:54.913677 10364 leader_election.cc:336] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105): Network error: Client connection negotiation failed: client connection to 127.5.142.133:33105: connect: Connection refused (error 111)
I20260812 08:01:54.913846 12074 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 25.
I20260812 08:01:54.914196 10364 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22
I20260812 08:01:54.914247 10071 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 25.
I20260812 08:01:54.914266  7378 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 25.
I20260812 08:01:54.914408 12278 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 25 FOLLOWER]: Leader election won for term 25
I20260812 08:01:54.914605 12278 raft_consensus.cc:697] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 25 LEADER]: Becoming Leader. State: Replica: 3ae3f7a6e9f943e996a6dcea504878d0, State: Running, Role: LEADER
I20260812 08:01:54.914772 12278 consensus_queue.cc:237] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3549, Committed index: 3549, Last appended: 24.3557, Last appended by leader: 3557, Current term: 25, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:54.916797  6745 catalog_manager.cc:5719] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 reported cstate change: term changed from 24 to 25, leader changed from 1664e32d3cd0473592085aa522379f22 (127.5.142.133) to 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129). New cstate: current_term: 25 leader_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } health_report { overall_health: UNKNOWN } } }
I20260812 08:01:55.213704  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.133:33105
--local_ip_for_outbound_sockets=127.5.142.133
--tserver_master_addrs=127.5.142.190:44379
--webserver_port=36547
--webserver_interface=127.5.142.133
--builtin_ntp_servers=127.5.142.148:43055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
W20260812 08:01:55.325269 12288 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:01:55.325496 12288 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:01:55.325557 12288 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260812 08:01:55.327658  8008 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 25 FOLLOWER]: Refusing update from remote peer 3ae3f7a6e9f943e996a6dcea504878d0: Log matching property violated. Preceding OpId in replica: term: 24 index: 3488. Preceding OpId from leader: term: 25 index: 3558. (index mismatch)
I20260812 08:01:55.328133 12278 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3558, Last known committed idx: 3483, Time since last communication: 0.000s
W20260812 08:01:55.328979 12288 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:01:55.329098 12288 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.133
W20260812 08:01:55.332300 10364 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.142.133:33105: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260812 08:01:55.333151 10364 consensus_peers.cc:597] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 -> Peer 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105): Couldn't send request to peer 1664e32d3cd0473592085aa522379f22. Status: Network error: Client connection negotiation failed: client connection to 127.5.142.133:33105: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 08:01:55.333776 12288 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:43055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.133:33105
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.5.142.133
--webserver_port=36547
--tserver_master_addrs=127.5.142.190:44379
--never_fsync=true
--heap_profile_path=/tmp/kudu.12288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.133

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:01:55.335242 12288 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:01:55.336719 12288 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:01:55.358695 12308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:55.358944 12306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:01:55.359082 12305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:01:55.363090  8220 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 25 FOLLOWER]: Refusing update from remote peer 3ae3f7a6e9f943e996a6dcea504878d0: Log matching property violated. Preceding OpId in replica: term: 24 index: 3550. Preceding OpId from leader: term: 25 index: 3558. (index mismatch)
I20260812 08:01:55.363243 12288 server_base.cc:1061] running on GCE node
I20260812 08:01:55.363818 12288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:01:55.364063 12278 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3558, Last known committed idx: 3542, Time since last communication: 0.000s
I20260812 08:01:55.364578 12288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:01:55.365758 12288 hybrid_clock.cc:648] HybridClock initialized: now 1786521715365720 us; error 51 us; skew 500 ppm
I20260812 08:01:55.368093 12288 webserver.cc:533] Webserver started at http://127.5.142.133:36547/ using document root <none> and password file <none>
I20260812 08:01:55.368860 12288 fs_manager.cc:362] Metadata directory not provided
I20260812 08:01:55.368968 12288 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:01:55.373237 12288 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:01:55.375227 12325 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:01:55.376276 12288 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:01:55.376377 12288 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "1664e32d3cd0473592085aa522379f22"
format_stamp: "Formatted at 2026-08-12 08:00:54 on dist-test-slave-ncp9"
I20260812 08:01:55.376761 12288 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:01:55.392318 12288 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:01:55.393000 12288 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:01:55.393157 12288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:01:55.393734 12288 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:01:55.395033 12344 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:01:55.398403 12288 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:01:55.398463 12288 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 08:01:55.398535 12288 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:01:55.400040 12288 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:01:55.400085 12288 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 08:01:55.400404 12344 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Bootstrap starting.
I20260812 08:01:55.410571 12288 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.133:33105
I20260812 08:01:55.410614 12499 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.133:33105 every 8 connection(s)
I20260812 08:01:55.411569 12288 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20260812 08:01:55.411898  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 12288
I20260812 08:01:55.421296 12504 heartbeater.cc:344] Connected to a master server at 127.5.142.190:44379
I20260812 08:01:55.421586 12504 heartbeater.cc:461] Registering TS with master...
I20260812 08:01:55.421923 12074 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 25 FOLLOWER]: Refusing update from remote peer 3ae3f7a6e9f943e996a6dcea504878d0: Log matching property violated. Preceding OpId in replica: term: 23 index: 3403. Preceding OpId from leader: term: 25 index: 3558. (index mismatch)
I20260812 08:01:55.422220 12504 heartbeater.cc:507] Master 127.5.142.190:44379 requested a full tablet report, sending...
I20260812 08:01:55.422566 12278 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3558, Last known committed idx: 3395, Time since last communication: 0.000s
I20260812 08:01:55.422717 12278 consensus_queue.cc:1243] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Peer 56b60424ed9f4a079689ad82a318d29a log is divergent from this leader: its last log entry 23.3403 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3395
I20260812 08:01:55.424103  6746 ts_manager.cc:194] Re-registered known tserver with Master: 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105)
I20260812 08:01:55.424834  6746 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.133:59597
I20260812 08:01:55.428128 12074 pending_rounds.cc:85] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting all ops after (but not including) 3395
I20260812 08:01:55.428308 12074 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting uncommitted WRITE_OP operation due to leader change: 23.3396
I20260812 08:01:55.428870 12074 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting uncommitted WRITE_OP operation due to leader change: 23.3397
I20260812 08:01:55.429178 12074 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting uncommitted WRITE_OP operation due to leader change: 23.3398
I20260812 08:01:55.429427 12074 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting uncommitted WRITE_OP operation due to leader change: 23.3399
I20260812 08:01:55.429615 12074 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting uncommitted WRITE_OP operation due to leader change: 23.3400
I20260812 08:01:55.429797 12074 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting uncommitted WRITE_OP operation due to leader change: 23.3401
I20260812 08:01:55.430069 12074 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting uncommitted WRITE_OP operation due to leader change: 23.3402
I20260812 08:01:55.430325 12074 pending_rounds.cc:107] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a: Aborting uncommitted WRITE_OP operation due to leader change: 23.3403
I20260812 08:01:55.437844 10071 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 25 FOLLOWER]: Refusing update from remote peer 3ae3f7a6e9f943e996a6dcea504878d0: Log matching property violated. Preceding OpId in replica: term: 24 index: 3557. Preceding OpId from leader: term: 25 index: 3558. (index mismatch)
I20260812 08:01:55.438406 12278 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3558, Last known committed idx: 3549, Time since last communication: 0.000s
I20260812 08:01:55.443926  7378 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 25 FOLLOWER]: Refusing update from remote peer 3ae3f7a6e9f943e996a6dcea504878d0: Log matching property violated. Preceding OpId in replica: term: 24 index: 3514. Preceding OpId from leader: term: 25 index: 3558. (index mismatch)
I20260812 08:01:55.451592 12278 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3558, Last known committed idx: 3513, Time since last communication: 0.000s
I20260812 08:01:55.467320 12536 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3531, attempt_no=1}) took 3115 ms. Trace:
I20260812 08:01:55.467527 12536 rpcz_store.cc:276] 0812 08:01:52.351346 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:52.351686 (+   340us) service_pool.cc:224] Handling call
0812 08:01:55.467296 (+3115610us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:01:55.470541 12536 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3532, attempt_no=1}) took 3119 ms. Trace:
I20260812 08:01:55.470695 12536 rpcz_store.cc:276] 0812 08:01:52.351394 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:52.351760 (+   366us) service_pool.cc:224] Handling call
0812 08:01:55.470527 (+3118767us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:01:55.483912 12535 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3536, attempt_no=1}) took 3132 ms. Trace:
I20260812 08:01:55.484020 12535 rpcz_store.cc:276] 0812 08:01:52.351292 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:52.352675 (+  1383us) service_pool.cc:224] Handling call
0812 08:01:55.483891 (+3131216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:01:55.487021 12531 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3535, attempt_no=1}) took 3135 ms. Trace:
I20260812 08:01:55.501036 12531 rpcz_store.cc:276] 0812 08:01:52.351441 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:52.351482 (+    41us) service_pool.cc:224] Handling call
0812 08:01:55.486990 (+3135508us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:01:55.501986 12535 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3538, attempt_no=1}) took 3150 ms. Trace:
I20260812 08:01:55.502714 12535 rpcz_store.cc:276] 0812 08:01:52.351590 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:52.351651 (+    61us) service_pool.cc:224] Handling call
0812 08:01:55.501972 (+3150321us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:01:55.503432 12529 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3537, attempt_no=1}) took 3151 ms. Trace:
I20260812 08:01:55.505733 12529 rpcz_store.cc:276] 0812 08:01:52.351542 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:52.351593 (+    51us) service_pool.cc:224] Handling call
0812 08:01:55.503399 (+3151806us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:01:55.507771 12536 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3533, attempt_no=1}) took 3156 ms. Trace:
I20260812 08:01:55.508231 12536 rpcz_store.cc:276] 0812 08:01:52.351494 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:52.351781 (+   287us) service_pool.cc:224] Handling call
0812 08:01:55.507759 (+3155978us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 08:01:55.505662 12531 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3534, attempt_no=1}) took 3154 ms. Trace:
I20260812 08:01:55.508641 12531 rpcz_store.cc:276] 0812 08:01:52.351650 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:52.351700 (+    50us) service_pool.cc:224] Handling call
0812 08:01:55.505648 (+3153948us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6528}
I20260812 08:01:55.640554 12519 mvcc.cc:204] Tried to move back new op lower bound from 7317592931902148608 to 7317592926471323648. Current Snapshot: MvccSnapshot[applied={T|T < 7317592931891396608}]
I20260812 08:01:55.696488 12344 log.cc:826] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Log is configured to *not* fsync() on all Append() calls
I20260812 08:01:56.427356 12504 heartbeater.cc:499] Master 127.5.142.190:44379 was elected leader, sending a full tablet report...
I20260812 08:01:57.563500 12578 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3ae3f7a6e9f943e996a6dcea504878d0)
I20260812 08:01:57.563650 12578 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:57.564363 12578 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183)
I20260812 08:01:57.564859  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 26 candidate_status { last_received { term: 25 index: 3676 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:57.564951 10073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 26 candidate_status { last_received { term: 25 index: 3676 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:57.565644 12075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 26 candidate_status { last_received { term: 25 index: 3676 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:57.565876  7380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 26 candidate_status { last_received { term: 25 index: 3676 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:57.566341  7907 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 780d7e64a8214418a371f35351b113df; no voters: 16901c5559a748a9aa770d73e254099c, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:57.566567 12578 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260812 08:01:57.569485 12437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 26 candidate_status { last_received { term: 25 index: 3676 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
W20260812 08:01:57.570673  7902 leader_election.cc:343] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 26 pre-election: Tablet error from VoteRequest() call to peer 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:01:57.787060 12585 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3ae3f7a6e9f943e996a6dcea504878d0)
I20260812 08:01:57.787204 12585 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:57.788239 10073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 3734 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:01:57.788499 12585 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:57.789045  7380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 3734 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:57.789397  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 3734 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:57.788812 12075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 3734 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:57.789549  8008 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6801d45ca6614156906e5d571ebcb8a6 in term 25.
I20260812 08:01:57.793097 12437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 3734 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
W20260812 08:01:57.793453  8116 leader_election.cc:343] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 26 pre-election: Tablet error from VoteRequest() call to peer 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:01:57.793529  8116 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 56b60424ed9f4a079689ad82a318d29a, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:01:57.793709 12585 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260812 08:01:57.794421 12589 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3ae3f7a6e9f943e996a6dcea504878d0)
I20260812 08:01:57.794529 12589 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:57.795233 12589 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:57.795591  7380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 26 candidate_status { last_received { term: 25 index: 3740 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:57.795771 12592 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3ae3f7a6e9f943e996a6dcea504878d0)
I20260812 08:01:57.795936 12592 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:57.796280  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 26 candidate_status { last_received { term: 25 index: 3740 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:57.796020 12075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 26 candidate_status { last_received { term: 25 index: 3740 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:01:57.796407  8225 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 25.
I20260812 08:01:57.796496 12075 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 25.
I20260812 08:01:57.796968  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 26 candidate_status { last_received { term: 25 index: 3740 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:57.797112  8008 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 25.
I20260812 08:01:57.797591  9966 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df, faafbc72cbd444ff90977a2cdb091b72; no voters: 16901c5559a748a9aa770d73e254099c
I20260812 08:01:57.798768 12589 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260812 08:01:57.798830 12589 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 3ae3f7a6e9f943e996a6dcea504878d0)
I20260812 08:01:57.798870 12589 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 25 FOLLOWER]: Advancing to term 26
I20260812 08:01:57.799322 12592 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:57.800057 12589 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:57.801015  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 26 candidate_status { last_received { term: 25 index: 3740 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
I20260812 08:01:57.801151  8225 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 25 FOLLOWER]: Advancing to term 26
I20260812 08:01:57.801339 12075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 26 candidate_status { last_received { term: 25 index: 3740 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a"
I20260812 08:01:57.801451 12075 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 25 FOLLOWER]: Advancing to term 26
I20260812 08:01:57.802175  8225 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 26.
I20260812 08:01:57.802501 12075 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 26.
I20260812 08:01:57.804207  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 26 candidate_status { last_received { term: 25 index: 3740 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:01:57.804334  8008 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 25 FOLLOWER]: Advancing to term 26
I20260812 08:01:57.804459  7380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 26 candidate_status { last_received { term: 25 index: 3740 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:01:57.805384  8008 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faafbc72cbd444ff90977a2cdb091b72 in term 26.
I20260812 08:01:57.805783  9965 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df, faafbc72cbd444ff90977a2cdb091b72; no voters: 16901c5559a748a9aa770d73e254099c
I20260812 08:01:57.806015  7380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 26 candidate_status { last_received { term: 25 index: 3740 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:01:57.803596 12589 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 26 election: Requested vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:01:57.806178 12589 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 26 FOLLOWER]: Leader election won for term 26
I20260812 08:01:57.806238 12589 raft_consensus.cc:697] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 26 LEADER]: Becoming Leader. State: Replica: faafbc72cbd444ff90977a2cdb091b72, State: Running, Role: LEADER
I20260812 08:01:57.806452 12589 consensus_queue.cc:237] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3732, Committed index: 3732, Last appended: 25.3740, Last appended by leader: 3740, Current term: 26, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:57.809826 12437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 26 candidate_status { last_received { term: 25 index: 3740 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:01:57.809885  8008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 26 candidate_status { last_received { term: 25 index: 3740 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:01:57.810037  8008 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56b60424ed9f4a079689ad82a318d29a in current term 26: Already voted for candidate faafbc72cbd444ff90977a2cdb091b72 in this term.
I20260812 08:01:57.810117 12436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 26 candidate_status { last_received { term: 25 index: 3740 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
W20260812 08:01:57.810285 11963 leader_election.cc:343] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 26 pre-election: Tablet error from VoteRequest() call to peer 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:01:57.810386 12435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 26 candidate_status { last_received { term: 25 index: 3740 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22"
W20260812 08:01:57.810868  9966 leader_election.cc:343] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 26 pre-election: Tablet error from VoteRequest() call to peer 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105): Illegal state: must be running to vote when last-logged opid is not known
W20260812 08:01:57.811010  9966 leader_election.cc:343] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [CANDIDATE]: Term 26 election: Tablet error from VoteRequest() call to peer 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105): Illegal state: must be running to vote when last-logged opid is not known
I20260812 08:01:57.811488  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 26 candidate_status { last_received { term: 25 index: 3740 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:01:57.811661  8225 raft_consensus.cc:2393] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56b60424ed9f4a079689ad82a318d29a in current term 26: Already voted for candidate faafbc72cbd444ff90977a2cdb091b72 in this term.
I20260812 08:01:57.812043 11964 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 56b60424ed9f4a079689ad82a318d29a; no voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6, 780d7e64a8214418a371f35351b113df
I20260812 08:01:57.812318 12592 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260812 08:01:57.812570  6745 catalog_manager.cc:5719] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 reported cstate change: term changed from 25 to 26, leader changed from 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129) to faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132). New cstate: current_term: 26 leader_uuid: "faafbc72cbd444ff90977a2cdb091b72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } health_report { overall_health: UNKNOWN } } }
I20260812 08:01:57.816179 10073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 26 candidate_status { last_received { term: 25 index: 3740 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
W20260812 08:01:57.991498 10363 connection.cc:256] RPC call timeout handler was delayed by -1.0293442821130157s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 08:01:57.993719 10469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "6801d45ca6614156906e5d571ebcb8a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 3734 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:57.994061 10470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 26 candidate_status { last_received { term: 25 index: 3740 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0"
I20260812 08:01:57.994515 10472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 26 candidate_status { last_received { term: 25 index: 3676 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:57.995118 10476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "faafbc72cbd444ff90977a2cdb091b72" candidate_term: 26 candidate_status { last_received { term: 25 index: 3740 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:57.995297  8008 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 26 FOLLOWER]: Rejecting Update request from peer 3ae3f7a6e9f943e996a6dcea504878d0 for earlier term 25. Current term is 26. Ops: [25.3677-25.3740]
I20260812 08:01:57.996301 12617 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 }, Status: INVALID_TERM, Last received: 25.3676, Next index: 3677, Last known committed idx: 3676, Time since last communication: 0.000s
I20260812 08:01:57.996436 12617 raft_consensus.cc:3055] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 25 LEADER]: Stepping down as leader of term 25
I20260812 08:01:57.996474 12617 raft_consensus.cc:740] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 3ae3f7a6e9f943e996a6dcea504878d0, State: Running, Role: LEADER
I20260812 08:01:57.996603 12617 consensus_queue.cc:260] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3734, Committed index: 3734, Last appended: 25.3740, Last appended by leader: 3740, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:57.996819 12617 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 25 FOLLOWER]: Advancing to term 26
I20260812 08:01:57.998118 12535 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3713, attempt_no=0}) took 1728 ms. Trace:
I20260812 08:01:57.998188 12535 rpcz_store.cc:276] 0812 08:01:56.269700 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:56.269972 (+   272us) service_pool.cc:224] Handling call
0812 08:01:57.998088 (+1728116us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:56.279872 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:56.279953 (+    81us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:56.279957 (+     4us) write_op.cc:435] Acquired schema lock
0812 08:01:56.279958 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:56.280042 (+    84us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:56.280047 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:56.280050 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:56.280109 (+    59us) tablet.cc:701] Row locks acquired
0812 08:01:56.280110 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:56.280138 (+    28us) write_op.cc:270] Start()
0812 08:01:56.280155 (+    17us) write_op.cc:276] Timestamp: P: 1786521716280093 usec, L: 0
0812 08:01:56.280167 (+    12us) op_driver.cc:348] REPLICATION: starting
0812 08:01:56.280270 (+   103us) log.cc:844] Serialized 696 byte log entry
0812 08:01:57.993426 (+1713156us) op_driver.cc:464] REPLICATION: finished
0812 08:01:57.997478 (+  4052us) write_op.cc:301] APPLY: starting
0812 08:01:57.997494 (+    16us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:57.997535 (+    41us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:57.997547 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:57.997744 (+   197us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:57.997780 (+    36us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:57.997785 (+     5us) write_op.cc:312] APPLY: finished
0812 08:01:57.997847 (+    62us) log.cc:844] Serialized 77 byte log entry
0812 08:01:57.998006 (+   159us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:57.998020 (+    14us) write_op.cc:454] Released schema lock
0812 08:01:57.998027 (+     7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3995,"num_ops":10,"prepare.queue_time_us":9646,"prepare.run_cpu_time_us":482,"prepare.run_wall_time_us":3386,"raft.queue_time_us":1826,"raft.run_cpu_time_us":6,"raft.run_wall_time_us":7,"replication_time_us":1713224,"thread_start_us":85,"threads_started":1,"wal-append.queue_time_us":260}]]}
I20260812 08:01:57.998994 12535 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3714, attempt_no=0}) took 1727 ms. Trace:
I20260812 08:01:57.999048 12535 rpcz_store.cc:276] 0812 08:01:56.271623 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:56.272042 (+   419us) service_pool.cc:224] Handling call
0812 08:01:57.998984 (+1726942us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:56.283292 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:56.283744 (+   452us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:56.283747 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:56.283749 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:56.283839 (+    90us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:56.283845 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:56.283848 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:56.283906 (+    58us) tablet.cc:701] Row locks acquired
0812 08:01:56.283907 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:56.283933 (+    26us) write_op.cc:270] Start()
0812 08:01:56.283954 (+    21us) write_op.cc:276] Timestamp: P: 1786521716283888 usec, L: 0
0812 08:01:56.283956 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:56.284054 (+    98us) log.cc:844] Serialized 696 byte log entry
0812 08:01:57.993470 (+1709416us) op_driver.cc:464] REPLICATION: finished
0812 08:01:57.998567 (+  5097us) write_op.cc:301] APPLY: starting
0812 08:01:57.998578 (+    11us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:57.998604 (+    26us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:57.998615 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:57.998790 (+   175us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:57.998818 (+    28us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:57.998823 (+     5us) write_op.cc:312] APPLY: finished
0812 08:01:57.998874 (+    51us) log.cc:844] Serialized 77 byte log entry
0812 08:01:57.998921 (+    47us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:57.998931 (+    10us) write_op.cc:454] Released schema lock
0812 08:01:57.998936 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":5071,"mutex_wait_us":49,"num_ops":10,"prepare.queue_time_us":10946,"prepare.run_cpu_time_us":539,"prepare.run_wall_time_us":1376,"raft.queue_time_us":120,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":277,"replication_time_us":1709503,"spinlock_wait_cycles":1408}]]}
I20260812 08:01:57.999435 10073 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 26 LEADER]: Rejecting Update request from peer 3ae3f7a6e9f943e996a6dcea504878d0 for earlier term 25. Current term is 26. Ops: []
I20260812 08:01:58.000247 12075 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 26 FOLLOWER]: Rejecting Update request from peer 3ae3f7a6e9f943e996a6dcea504878d0 for earlier term 25. Current term is 26. Ops: []
I20260812 08:01:58.002413  8225 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 26 FOLLOWER]: Rejecting Update request from peer 3ae3f7a6e9f943e996a6dcea504878d0 for earlier term 25. Current term is 26. Ops: [25.3735-25.3740]
W20260812 08:01:58.004292 10412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:01:58.004482 10420 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
I20260812 08:01:58.005323 10476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 26 candidate_status { last_received { term: 25 index: 3740 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:01:58.005451 10476 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 26.
I20260812 08:01:58.014384 12075 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 26 FOLLOWER]: Refusing update from remote peer faafbc72cbd444ff90977a2cdb091b72: Log matching property violated. Preceding OpId in replica: term: 25 index: 3740. Preceding OpId from leader: term: 26 index: 3742. (index mismatch)
I20260812 08:01:58.014393  8008 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 26 FOLLOWER]: Refusing update from remote peer faafbc72cbd444ff90977a2cdb091b72: Log matching property violated. Preceding OpId in replica: term: 25 index: 3676. Preceding OpId from leader: term: 26 index: 3742. (index mismatch)
I20260812 08:01:58.014887 12610 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3741, Last known committed idx: 3732, Time since last communication: 0.000s
I20260812 08:01:58.014964  8225 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 26 FOLLOWER]: Refusing update from remote peer faafbc72cbd444ff90977a2cdb091b72: Log matching property violated. Preceding OpId in replica: term: 25 index: 3734. Preceding OpId from leader: term: 26 index: 3742. (index mismatch)
I20260812 08:01:58.015681 12589 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3741, Last known committed idx: 3732, Time since last communication: 0.000s
I20260812 08:01:58.016033 12589 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3741, Last known committed idx: 3676, Time since last communication: 0.000s
I20260812 08:01:58.017794 10476 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 26 FOLLOWER]: Refusing update from remote peer faafbc72cbd444ff90977a2cdb091b72: Log matching property violated. Preceding OpId in replica: term: 25 index: 3740. Preceding OpId from leader: term: 26 index: 3742. (index mismatch)
W20260812 08:01:58.018189  9966 consensus_peers.cc:597] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 -> Peer 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105): Couldn't send request to peer 1664e32d3cd0473592085aa522379f22. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:01:58.018806 12610 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3741, Last known committed idx: 3734, Time since last communication: 0.000s
I20260812 08:01:58.028867  7378 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 25 FOLLOWER]: Advancing to term 26
I20260812 08:01:58.030128  7378 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 26 FOLLOWER]: Refusing update from remote peer faafbc72cbd444ff90977a2cdb091b72: Log matching property violated. Preceding OpId in replica: term: 25 index: 3740. Preceding OpId from leader: term: 26 index: 3742. (index mismatch)
I20260812 08:01:58.031237 12589 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3741, Last known committed idx: 3734, Time since last communication: 0.000s
I20260812 08:01:58.040009 12536 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3715, attempt_no=0}) took 1766 ms. Trace:
I20260812 08:01:58.040095 12536 rpcz_store.cc:276] 0812 08:01:56.273113 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:56.273165 (+    52us) service_pool.cc:224] Handling call
0812 08:01:58.039992 (+1766827us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:56.284697 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:56.285998 (+  1301us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:56.286001 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:56.286003 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:56.286086 (+    83us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:56.286091 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:56.286094 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:56.286169 (+    75us) tablet.cc:701] Row locks acquired
0812 08:01:56.286171 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:56.286199 (+    28us) write_op.cc:270] Start()
0812 08:01:56.286218 (+    19us) write_op.cc:276] Timestamp: P: 1786521716286153 usec, L: 0
0812 08:01:56.286220 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:56.286320 (+   100us) log.cc:844] Serialized 686 byte log entry
0812 08:01:58.038833 (+1752513us) op_driver.cc:464] REPLICATION: finished
0812 08:01:58.039501 (+   668us) write_op.cc:301] APPLY: starting
0812 08:01:58.039517 (+    16us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:58.039552 (+    35us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:58.039564 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:58.039768 (+   204us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:58.039797 (+    29us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:58.039801 (+     4us) write_op.cc:312] APPLY: finished
0812 08:01:58.039859 (+    58us) log.cc:844] Serialized 77 byte log entry
0812 08:01:58.039905 (+    46us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:58.039924 (+    19us) write_op.cc:454] Released schema lock
0812 08:01:58.039930 (+     6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":627,"num_ops":10,"prepare.queue_time_us":11276,"prepare.run_cpu_time_us":474,"prepare.run_wall_time_us":1737,"raft.queue_time_us":27,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":4,"replication_time_us":1752587}]]}
I20260812 08:01:58.040886 12536 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3716, attempt_no=0}) took 1763 ms. Trace:
I20260812 08:01:58.040970 12536 rpcz_store.cc:276] 0812 08:01:56.277535 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:56.277591 (+    56us) service_pool.cc:224] Handling call
0812 08:01:58.040875 (+1763284us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:56.286449 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:56.286518 (+    69us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:56.286521 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:56.286522 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:56.286625 (+   103us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:56.286630 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:56.286633 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:56.286691 (+    58us) tablet.cc:701] Row locks acquired
0812 08:01:56.286693 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:56.286720 (+    27us) write_op.cc:270] Start()
0812 08:01:56.287272 (+   552us) write_op.cc:276] Timestamp: P: 1786521716286674 usec, L: 0
0812 08:01:56.287276 (+     4us) op_driver.cc:348] REPLICATION: starting
0812 08:01:56.287375 (+    99us) log.cc:844] Serialized 696 byte log entry
0812 08:01:58.038898 (+1751523us) op_driver.cc:464] REPLICATION: finished
0812 08:01:58.040464 (+  1566us) write_op.cc:301] APPLY: starting
0812 08:01:58.040473 (+     9us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:58.040499 (+    26us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:58.040510 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:58.040690 (+   180us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:58.040718 (+    28us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:58.040722 (+     4us) write_op.cc:312] APPLY: finished
0812 08:01:58.040778 (+    56us) log.cc:844] Serialized 77 byte log entry
0812 08:01:58.040812 (+    34us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:58.040822 (+    10us) write_op.cc:454] Released schema lock
0812 08:01:58.040826 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1541,"num_ops":10,"prepare.queue_time_us":8610,"prepare.run_cpu_time_us":461,"prepare.run_wall_time_us":991,"raft.queue_time_us":720,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":4,"replication_time_us":1751608}]]}
I20260812 08:01:58.041702 12536 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3717, attempt_no=0}) took 1764 ms. Trace:
I20260812 08:01:58.041754 12536 rpcz_store.cc:276] 0812 08:01:56.277601 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:56.277659 (+    58us) service_pool.cc:224] Handling call
0812 08:01:58.041691 (+1764032us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:56.287463 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:56.287532 (+    69us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:56.287535 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:56.287536 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:56.287618 (+    82us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:56.287623 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:56.287626 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:56.287912 (+   286us) tablet.cc:701] Row locks acquired
0812 08:01:56.287914 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:56.287942 (+    28us) write_op.cc:270] Start()
0812 08:01:56.287960 (+    18us) write_op.cc:276] Timestamp: P: 1786521716287896 usec, L: 0
0812 08:01:56.287961 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:01:56.288057 (+    96us) log.cc:844] Serialized 696 byte log entry
0812 08:01:58.038949 (+1750892us) op_driver.cc:464] REPLICATION: finished
0812 08:01:58.041286 (+  2337us) write_op.cc:301] APPLY: starting
0812 08:01:58.041295 (+     9us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:58.041320 (+    25us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:58.041330 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:58.041507 (+   177us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:58.041533 (+    26us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:58.041537 (+     4us) write_op.cc:312] APPLY: finished
0812 08:01:58.041588 (+    51us) log.cc:844] Serialized 77 byte log entry
0812 08:01:58.041620 (+    32us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:58.041630 (+    10us) write_op.cc:454] Released schema lock
0812 08:01:58.041644 (+    14us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2313,"num_ops":10,"prepare.queue_time_us":9548,"prepare.run_cpu_time_us":461,"prepare.run_wall_time_us":738,"raft.queue_time_us":55,"raft.run_cpu_time_us":12,"raft.run_wall_time_us":2,"replication_time_us":1750978,"thread_start_us":74,"threads_started":1}]]}
I20260812 08:01:58.042469 12536 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3718, attempt_no=0}) took 1760 ms. Trace:
I20260812 08:01:58.042520 12536 rpcz_store.cc:276] 0812 08:01:56.282008 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:56.282091 (+    83us) service_pool.cc:224] Handling call
0812 08:01:58.042458 (+1760367us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:56.288219 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:56.288283 (+    64us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:56.288286 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:56.288288 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:56.288357 (+    69us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:56.288362 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:56.288364 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:56.288418 (+    54us) tablet.cc:701] Row locks acquired
0812 08:01:56.288419 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:56.288442 (+    23us) write_op.cc:270] Start()
0812 08:01:56.288459 (+    17us) write_op.cc:276] Timestamp: P: 1786521716288398 usec, L: 0
0812 08:01:56.288461 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:56.288560 (+    99us) log.cc:844] Serialized 696 byte log entry
0812 08:01:58.039063 (+1750503us) op_driver.cc:464] REPLICATION: finished
0812 08:01:58.042056 (+  2993us) write_op.cc:301] APPLY: starting
0812 08:01:58.042066 (+    10us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:58.042091 (+    25us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:58.042102 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:58.042280 (+   178us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:58.042307 (+    27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:58.042312 (+     5us) write_op.cc:312] APPLY: finished
0812 08:01:58.042362 (+    50us) log.cc:844] Serialized 77 byte log entry
0812 08:01:58.042393 (+    31us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:58.042406 (+    13us) write_op.cc:454] Released schema lock
0812 08:01:58.042410 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2968,"num_ops":10,"prepare.queue_time_us":5851,"prepare.run_cpu_time_us":412,"prepare.run_wall_time_us":452,"raft.queue_time_us":27,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":5,"replication_time_us":1750590,"thread_start_us":78,"threads_started":1}]]}
I20260812 08:01:58.047133 12535 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3720, attempt_no=0}) took 1755 ms. Trace:
I20260812 08:01:58.047217 12535 rpcz_store.cc:276] 0812 08:01:56.292122 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:56.292176 (+    54us) service_pool.cc:224] Handling call
0812 08:01:58.047120 (+1754944us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:56.292464 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:56.292527 (+    63us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:56.292529 (+     2us) write_op.cc:435] Acquired schema lock
0812 08:01:56.292530 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:56.292609 (+    79us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:56.292614 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:56.292617 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:56.292674 (+    57us) tablet.cc:701] Row locks acquired
0812 08:01:56.292675 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:56.292704 (+    29us) write_op.cc:270] Start()
0812 08:01:56.292724 (+    20us) write_op.cc:276] Timestamp: P: 1786521716292658 usec, L: 0
0812 08:01:56.292725 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:01:56.292827 (+   102us) log.cc:844] Serialized 686 byte log entry
0812 08:01:58.039217 (+1746390us) op_driver.cc:464] REPLICATION: finished
0812 08:01:58.043297 (+  4080us) write_op.cc:301] APPLY: starting
0812 08:01:58.043308 (+    11us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:58.043335 (+    27us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:58.043347 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:58.043509 (+   162us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:58.043538 (+    29us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:58.043543 (+     5us) write_op.cc:312] APPLY: finished
0812 08:01:58.043595 (+    52us) log.cc:844] Serialized 77 byte log entry
0812 08:01:58.043647 (+    52us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:58.043661 (+    14us) write_op.cc:454] Released schema lock
0812 08:01:58.043666 (+     5us) write_op.cc:341] FINISH: Updating metrics
0812 08:01:58.047098 (+  3432us) spinlock_profiling.cc:243] Waited 3.22 ms on lock 0x563343bb4f18. stack: 00007fe58dfd0d35 00007fe58dfd1325 00007fe58dadd82d 00005633135a4b18 00005633135a4b6b 00005633135a89f2 00007fe58efcaf3e 00007fe58efd73c3 00007fe5911415fc 00007fe5905d518f 00007fe5905d4b95 00007fe5905d39ca 00007fe5905d695b 000056331363444d 00007fe58dff4326 00007fe58dff4c4e
Metrics: {"child_traces":[["op",{"apply.queue_time_us":4042,"num_ops":10,"prepare.queue_time_us":45,"prepare.run_cpu_time_us":465,"prepare.run_wall_time_us":871,"raft.queue_time_us":143,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":174,"replication_time_us":1746482,"spinlock_wait_cycles":7469440}]]}
I20260812 08:01:58.048157 12536 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:34500 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3719, attempt_no=0}) took 1766 ms. Trace:
I20260812 08:01:58.048481 12536 rpcz_store.cc:276] 0812 08:01:56.282087 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:56.282124 (+    37us) service_pool.cc:224] Handling call
0812 08:01:58.048143 (+1766019us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:56.288686 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:56.288754 (+    68us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:56.288757 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:56.288759 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:56.288840 (+    81us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:56.288846 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:56.288849 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:56.288915 (+    66us) tablet.cc:701] Row locks acquired
0812 08:01:56.288917 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:56.288945 (+    28us) write_op.cc:270] Start()
0812 08:01:56.288963 (+    18us) write_op.cc:276] Timestamp: P: 1786521716288900 usec, L: 0
0812 08:01:56.288965 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:56.289063 (+    98us) log.cc:844] Serialized 696 byte log entry
0812 08:01:58.039186 (+1750123us) op_driver.cc:464] REPLICATION: finished
0812 08:01:58.042845 (+  3659us) write_op.cc:301] APPLY: starting
0812 08:01:58.042854 (+     9us) tablet.cc:1367] starting BulkCheckPresence
0812 08:01:58.042879 (+    25us) tablet.cc:1370] finished BulkCheckPresence
0812 08:01:58.042889 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:01:58.043042 (+   153us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:01:58.043069 (+    27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:01:58.043074 (+     5us) write_op.cc:312] APPLY: finished
0812 08:01:58.043127 (+    53us) log.cc:844] Serialized 77 byte log entry
0812 08:01:58.043162 (+    35us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:01:58.043172 (+    10us) write_op.cc:454] Released schema lock
0812 08:01:58.043177 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3632,"num_ops":10,"prepare.queue_time_us":6306,"prepare.run_cpu_time_us":465,"prepare.run_wall_time_us":565,"raft.queue_time_us":51,"raft.run_cpu_time_us":38,"raft.run_wall_time_us":38,"replication_time_us":1750207,"spinlock_wait_cycles":57728}]]}
I20260812 08:01:58.084961 12650 mvcc.cc:204] Tried to move back new op lower bound from 7317592956980084736 to 7317592956143427584. Current Snapshot: MvccSnapshot[applied={T|T < 7317592956980084736 or (T in {7317592956980084736})}]
I20260812 08:01:58.873605 12344 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Bootstrap replayed 1/1 log segments. Stats: ops{read=3580 overwritten=23 applied=3549 ignored=0} inserts{seen=35310 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 8 replicates
I20260812 08:01:58.874413 12344 tablet_bootstrap.cc:492] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Bootstrap complete.
I20260812 08:01:58.876042 12344 ts_tablet_manager.cc:1403] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Time spent bootstrapping tablet: real 3.476s	user 2.837s	sys 0.157s
I20260812 08:01:58.878159 12344 raft_consensus.cc:359] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 24 FOLLOWER]: Replica starting. Triggering 8 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:58.881465 12344 raft_consensus.cc:740] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 24 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1664e32d3cd0473592085aa522379f22, State: Initialized, Role: FOLLOWER
I20260812 08:01:58.882149 12344 consensus_queue.cc:260] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3549, Last appended: 24.3557, Last appended by leader: 3557, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:01:58.883237 12344 ts_tablet_manager.cc:1434] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260812 08:01:59.121922 12435 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 24 FOLLOWER]: Advancing to term 26
I20260812 08:01:59.123992 12435 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 26 FOLLOWER]: Refusing update from remote peer faafbc72cbd444ff90977a2cdb091b72: Log matching property violated. Preceding OpId in replica: term: 24 index: 3557. Preceding OpId from leader: term: 25 index: 3740. (index mismatch)
I20260812 08:01:59.125182 12664 consensus_queue.cc:1050] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Got LMP mismatch error from peer: Peer: permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3741, Last known committed idx: 3549, Time since last communication: 0.000s
I20260812 08:01:59.480396 12721 mvcc.cc:204] Tried to move back new op lower bound from 7317592960725159936 to 7317592944293556224. Current Snapshot: MvccSnapshot[applied={T|T < 7317592959238713344}]
I20260812 08:02:00.212677 12736 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 26 FOLLOWER]: Starting pre-election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:02:00.212836 12736 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:02:00.213721 12736 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183)
I20260812 08:02:00.214180  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 27 candidate_status { last_received { term: 26 index: 3872 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:02:00.214180  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 27 candidate_status { last_received { term: 26 index: 3872 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:02:00.214198 12435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 27 candidate_status { last_received { term: 26 index: 3872 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:02:00.214659 12074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 27 candidate_status { last_received { term: 26 index: 3872 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:02:00.214811 10476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 27 candidate_status { last_received { term: 26 index: 3872 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:02:00.215070  7902 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 56b60424ed9f4a079689ad82a318d29a, 6801d45ca6614156906e5d571ebcb8a6
I20260812 08:02:00.215271 12736 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260812 08:02:00.446231 12738 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:02:00.446393 12738 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:02:00.447392 12738 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:02:00.447779  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 27 candidate_status { last_received { term: 26 index: 3945 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:02:00.447795 12435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 27 candidate_status { last_received { term: 26 index: 3945 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:02:00.447850  8003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 27 candidate_status { last_received { term: 26 index: 3945 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:02:00.447889 12074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 27 candidate_status { last_received { term: 26 index: 3945 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:02:00.447973  8003 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 26.
I20260812 08:02:00.448037 12074 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae3f7a6e9f943e996a6dcea504878d0 in term 26.
I20260812 08:02:00.448406 12744 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:02:00.448495 12744 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:02:00.448657  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 27 candidate_status { last_received { term: 26 index: 3945 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:02:00.449256 10476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 27 candidate_status { last_received { term: 26 index: 3945 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:02:00.449342 12435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 27 candidate_status { last_received { term: 26 index: 3945 } } ignore_live_leader: false dest_uuid: "1664e32d3cd0473592085aa522379f22" is_pre_election: true
I20260812 08:02:00.449371 12744 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 1664e32d3cd0473592085aa522379f22 (127.5.142.133:33105), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:02:00.449384 10476 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 26.
I20260812 08:02:00.449714  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 27 candidate_status { last_received { term: 26 index: 3945 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:02:00.449909  8003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 27 candidate_status { last_received { term: 26 index: 3945 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:02:00.449967  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 27 candidate_status { last_received { term: 26 index: 3945 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:02:00.450047  8003 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b60424ed9f4a079689ad82a318d29a in term 26.
I20260812 08:02:00.634989 12745 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:02:00.635180 12745 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:02:00.637717 12745 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:02:00.641821  8003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 27 candidate_status { last_received { term: 26 index: 3945 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df" is_pre_election: true
I20260812 08:02:00.642011  8003 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 26.
I20260812 08:02:00.642028 10476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 27 candidate_status { last_received { term: 26 index: 3945 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" is_pre_election: true
I20260812 08:02:00.642171 10476 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 26.
I20260812 08:02:00.644209  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 27 candidate_status { last_received { term: 26 index: 3945 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6" is_pre_election: true
I20260812 08:02:00.644606 12074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 27 candidate_status { last_received { term: 26 index: 3945 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a" is_pre_election: true
I20260812 08:02:00.644763 12074 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 26.
I20260812 08:02:00.645102 12330 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, 780d7e64a8214418a371f35351b113df; no voters: 6801d45ca6614156906e5d571ebcb8a6
I20260812 08:02:00.645299  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 27 candidate_status { last_received { term: 26 index: 3945 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c" is_pre_election: true
I20260812 08:02:00.645398 12745 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260812 08:02:00.645464 12745 raft_consensus.cc:493] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 26 FOLLOWER]: Starting leader election (detected failure of leader faafbc72cbd444ff90977a2cdb091b72)
I20260812 08:02:00.645509 12745 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 26 FOLLOWER]: Advancing to term 27
I20260812 08:02:00.646425 12745 raft_consensus.cc:515] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:02:00.646991 12745 leader_election.cc:290] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 27 election: Requested vote from peers 3ae3f7a6e9f943e996a6dcea504878d0 (127.5.142.129:41863), faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132:43153), 56b60424ed9f4a079689ad82a318d29a (127.5.142.130:38363), 6801d45ca6614156906e5d571ebcb8a6 (127.5.142.135:42409), 16901c5559a748a9aa770d73e254099c (127.5.142.131:33183), 780d7e64a8214418a371f35351b113df (127.5.142.134:33021)
I20260812 08:02:00.647239 12074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 27 candidate_status { last_received { term: 26 index: 3945 } } ignore_live_leader: false dest_uuid: "56b60424ed9f4a079689ad82a318d29a"
I20260812 08:02:00.647275 10476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 27 candidate_status { last_received { term: 26 index: 3945 } } ignore_live_leader: false dest_uuid: "3ae3f7a6e9f943e996a6dcea504878d0"
I20260812 08:02:00.647361 12074 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 26 FOLLOWER]: Advancing to term 27
I20260812 08:02:00.647400 10476 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 26 FOLLOWER]: Advancing to term 27
I20260812 08:02:00.647396  7378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 27 candidate_status { last_received { term: 26 index: 3945 } } ignore_live_leader: false dest_uuid: "16901c5559a748a9aa770d73e254099c"
I20260812 08:02:00.647532  8003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 27 candidate_status { last_received { term: 26 index: 3945 } } ignore_live_leader: false dest_uuid: "780d7e64a8214418a371f35351b113df"
I20260812 08:02:00.647657  8003 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 26 FOLLOWER]: Advancing to term 27
I20260812 08:02:00.648108  8225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 27 candidate_status { last_received { term: 26 index: 3945 } } ignore_live_leader: false dest_uuid: "6801d45ca6614156906e5d571ebcb8a6"
I20260812 08:02:00.648435 12074 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 27.
I20260812 08:02:00.648440 10476 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 27.
I20260812 08:02:00.648658  8003 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 27.
I20260812 08:02:00.649003 12329 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 1664e32d3cd0473592085aa522379f22, 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, 780d7e64a8214418a371f35351b113df; no voters: 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6
I20260812 08:02:00.649169 12745 raft_consensus.cc:2804] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 27 FOLLOWER]: Leader election won for term 27
I20260812 08:02:00.649380 12745 raft_consensus.cc:697] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [term 27 LEADER]: Becoming Leader. State: Replica: 1664e32d3cd0473592085aa522379f22, State: Running, Role: LEADER
I20260812 08:02:00.649727 12745 consensus_queue.cc:237] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3937, Committed index: 3937, Last appended: 26.3945, Last appended by leader: 3945, Current term: 27, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:02:00.652431  6746 catalog_manager.cc:5719] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 reported cstate change: term changed from 26 to 27, leader changed from faafbc72cbd444ff90977a2cdb091b72 (127.5.142.132) to 1664e32d3cd0473592085aa522379f22 (127.5.142.133). New cstate: current_term: 27 leader_uuid: "1664e32d3cd0473592085aa522379f22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } health_report { overall_health: UNKNOWN } } }
W20260812 08:02:00.730078  9965 connection.cc:256] RPC call timeout handler was delayed by -1.1189333512447774s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 08:02:00.730453  9969 connection.cc:256] RPC call timeout handler was delayed by -1.0063575832173228s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 08:02:00.731212 10073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" candidate_term: 27 candidate_status { last_received { term: 26 index: 3945 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:02:00.731498 10071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "780d7e64a8214418a371f35351b113df" candidate_term: 27 candidate_status { last_received { term: 26 index: 3872 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:02:00.731717 10070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "56b60424ed9f4a079689ad82a318d29a" candidate_term: 27 candidate_status { last_received { term: 26 index: 3945 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:02:00.732648 10363 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:02:00.732869 12738 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260812 08:02:00.733155 10476 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 27 FOLLOWER]: Rejecting Update request from peer faafbc72cbd444ff90977a2cdb091b72 for earlier term 26. Current term is 27. Ops: []
I20260812 08:02:00.733202 11961 leader_election.cc:304] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 3ae3f7a6e9f943e996a6dcea504878d0, 56b60424ed9f4a079689ad82a318d29a, 780d7e64a8214418a371f35351b113df; no voters: 1664e32d3cd0473592085aa522379f22, 16901c5559a748a9aa770d73e254099c, 6801d45ca6614156906e5d571ebcb8a6, faafbc72cbd444ff90977a2cdb091b72
I20260812 08:02:00.734170 12712 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 }, Status: INVALID_TERM, Last received: 26.3945, Next index: 3946, Last known committed idx: 3937, Time since last communication: 0.000s
I20260812 08:02:00.735553 12074 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 27 FOLLOWER]: Rejecting Update request from peer faafbc72cbd444ff90977a2cdb091b72 for earlier term 26. Current term is 27. Ops: []
I20260812 08:02:00.736027  8003 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 27 FOLLOWER]: Rejecting Update request from peer faafbc72cbd444ff90977a2cdb091b72 for earlier term 26. Current term is 27. Ops: [26.3873-26.3945]
I20260812 08:02:00.736043 12779 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 }, Status: INVALID_TERM, Last received: 26.3945, Next index: 3946, Last known committed idx: 3937, Time since last communication: 0.000s
I20260812 08:02:00.736807 12779 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 }, Status: INVALID_TERM, Last received: 26.3872, Next index: 3873, Last known committed idx: 3870, Time since last communication: 0.000s
I20260812 08:02:00.737339 12712 raft_consensus.cc:3055] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 26 LEADER]: Stepping down as leader of term 26
I20260812 08:02:00.737360 12744 raft_consensus.cc:2749] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260812 08:02:00.737396 12712 raft_consensus.cc:740] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: faafbc72cbd444ff90977a2cdb091b72, State: Running, Role: LEADER
I20260812 08:02:00.737529 12712 consensus_queue.cc:260] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3938, Committed index: 3945, Last appended: 26.3945, Last appended by leader: 3945, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 } } peers { permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 } } peers { permanent_uuid: "1664e32d3cd0473592085aa522379f22" member_type: VOTER last_known_addr { host: "127.5.142.133" port: 33105 } } peers { permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 } } peers { permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 } } peers { permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 } } peers { permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 } }
I20260812 08:02:00.737646 12074 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 27 FOLLOWER]: Rejecting Update request from peer faafbc72cbd444ff90977a2cdb091b72 for earlier term 26. Current term is 27. Ops: []
I20260812 08:02:00.738284 10476 raft_consensus.cc:1240] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 27 FOLLOWER]: Rejecting Update request from peer faafbc72cbd444ff90977a2cdb091b72 for earlier term 26. Current term is 27. Ops: []
I20260812 08:02:00.738713 12779 consensus_queue.cc:1059] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 }, Status: INVALID_TERM, Last received: 26.3945, Next index: 3946, Last known committed idx: 3937, Time since last communication: 0.000s
I20260812 08:02:00.740803 12712 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 26 FOLLOWER]: Advancing to term 27
I20260812 08:02:00.741091 10065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 27 candidate_status { last_received { term: 26 index: 3945 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72"
I20260812 08:02:00.741992 12790 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59446 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3917, attempt_no=0}) took 1810 ms. Trace:
I20260812 08:02:00.742064 12790 rpcz_store.cc:276] 0812 08:01:58.931337 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:58.931386 (+    49us) service_pool.cc:224] Handling call
0812 08:02:00.741981 (+1810595us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:58.931708 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:58.931778 (+    70us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:58.931781 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:58.931783 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:58.933268 (+  1485us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:58.933275 (+     7us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:58.933278 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:58.933334 (+    56us) tablet.cc:701] Row locks acquired
0812 08:01:58.933335 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:58.933365 (+    30us) write_op.cc:270] Start()
0812 08:01:58.933383 (+    18us) write_op.cc:276] Timestamp: P: 1786521718933284 usec, L: 0
0812 08:01:58.933385 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:58.933522 (+   137us) log.cc:844] Serialized 696 byte log entry
0812 08:02:00.736408 (+1802886us) op_driver.cc:464] REPLICATION: finished
0812 08:02:00.741370 (+  4962us) write_op.cc:301] APPLY: starting
0812 08:02:00.741384 (+    14us) tablet.cc:1367] starting BulkCheckPresence
0812 08:02:00.741427 (+    43us) tablet.cc:1370] finished BulkCheckPresence
0812 08:02:00.741437 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:02:00.741619 (+   182us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:02:00.741653 (+    34us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:02:00.741658 (+     5us) write_op.cc:312] APPLY: finished
0812 08:02:00.741730 (+    72us) log.cc:844] Serialized 77 byte log entry
0812 08:02:00.741893 (+   163us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:02:00.741909 (+    16us) write_op.cc:454] Released schema lock
0812 08:02:00.741915 (+     6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":4917,"num_ops":10,"prepare.queue_time_us":60,"prepare.run_cpu_time_us":538,"prepare.run_wall_time_us":3076,"raft.queue_time_us":1039,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":175,"replication_time_us":1802992,"spinlock_wait_cycles":52480,"thread_start_us":177,"threads_started":2}]]}
I20260812 08:02:00.741989 10079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22cf4f6412248aa9e68c6d5d61153e8" candidate_uuid: "1664e32d3cd0473592085aa522379f22" candidate_term: 27 candidate_status { last_received { term: 26 index: 3945 } } ignore_live_leader: false dest_uuid: "faafbc72cbd444ff90977a2cdb091b72" is_pre_election: true
I20260812 08:02:00.742965 10065 raft_consensus.cc:2468] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1664e32d3cd0473592085aa522379f22 in term 27.
W20260812 08:02:00.744043 10000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Illegal state: replica faafbc72cbd444ff90977a2cdb091b72 is not leader of this config: current role FOLLOWER
I20260812 08:02:00.745108 12788 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59446 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3918, attempt_no=0}) took 1813 ms. Trace:
I20260812 08:02:00.746260 12788 rpcz_store.cc:276] 0812 08:01:58.931411 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:58.931474 (+    63us) service_pool.cc:224] Handling call
0812 08:02:00.745095 (+1813621us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:58.934822 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:58.934892 (+    70us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:58.934896 (+     4us) write_op.cc:435] Acquired schema lock
0812 08:01:58.934897 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:58.934975 (+    78us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:58.934981 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:58.934984 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:58.935041 (+    57us) tablet.cc:701] Row locks acquired
0812 08:01:58.935043 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:58.935069 (+    26us) write_op.cc:270] Start()
0812 08:01:58.935086 (+    17us) write_op.cc:276] Timestamp: P: 1786521718934989 usec, L: 0
0812 08:01:58.935088 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:58.935189 (+   101us) log.cc:844] Serialized 696 byte log entry
0812 08:02:00.736565 (+1801376us) op_driver.cc:464] REPLICATION: finished
0812 08:02:00.744613 (+  8048us) write_op.cc:301] APPLY: starting
0812 08:02:00.744628 (+    15us) tablet.cc:1367] starting BulkCheckPresence
0812 08:02:00.744666 (+    38us) tablet.cc:1370] finished BulkCheckPresence
0812 08:02:00.744679 (+    13us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:02:00.744860 (+   181us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:02:00.744911 (+    51us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:02:00.744916 (+     5us) write_op.cc:312] APPLY: finished
0812 08:02:00.744978 (+    62us) log.cc:844] Serialized 77 byte log entry
0812 08:02:00.745017 (+    39us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:02:00.745029 (+    12us) write_op.cc:454] Released schema lock
0812 08:02:00.745035 (+     6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":7989,"mutex_wait_us":687,"num_ops":10,"prepare.queue_time_us":3151,"prepare.run_cpu_time_us":457,"prepare.run_wall_time_us":1601,"raft.queue_time_us":1186,"raft.run_cpu_time_us":8,"raft.run_wall_time_us":8,"replication_time_us":1801459,"thread_start_us":54,"threads_started":1}]]}
I20260812 08:02:00.747478 12788 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59446 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3921, attempt_no=0}) took 1810 ms. Trace:
W20260812 08:02:00.753947 10420 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:02:00.755070 10420 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:02:00.760689  8145 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39074: Illegal state: replica 6801d45ca6614156906e5d571ebcb8a6 is not leader of this config: current role FOLLOWER
I20260812 08:02:00.762737 12788 rpcz_store.cc:276] 0812 08:01:58.937245 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:58.937832 (+   587us) service_pool.cc:224] Handling call
0812 08:02:00.747467 (+1809635us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:58.939379 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:58.939451 (+    72us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:58.939454 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:58.939456 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:58.939534 (+    78us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:58.939540 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:58.939543 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:58.939600 (+    57us) tablet.cc:701] Row locks acquired
0812 08:01:58.939602 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:58.939640 (+    38us) write_op.cc:270] Start()
0812 08:01:58.939659 (+    19us) write_op.cc:276] Timestamp: P: 1786521718939549 usec, L: 0
0812 08:01:58.939660 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:01:58.939746 (+    86us) log.cc:844] Serialized 696 byte log entry
0812 08:02:00.736872 (+1797126us) op_driver.cc:464] REPLICATION: finished
0812 08:02:00.747067 (+ 10195us) write_op.cc:301] APPLY: starting
0812 08:02:00.747090 (+    23us) tablet.cc:1367] starting BulkCheckPresence
0812 08:02:00.747117 (+    27us) tablet.cc:1370] finished BulkCheckPresence
0812 08:02:00.747127 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:02:00.747293 (+   166us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:02:00.747319 (+    26us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:02:00.747323 (+     4us) write_op.cc:312] APPLY: finished
0812 08:02:00.747375 (+    52us) log.cc:844] Serialized 77 byte log entry
0812 08:02:00.747407 (+    32us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:02:00.747418 (+    11us) write_op.cc:454] Released schema lock
0812 08:02:00.747422 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":10167,"num_ops":10,"prepare.queue_time_us":1323,"prepare.run_cpu_time_us":437,"prepare.run_wall_time_us":437,"raft.queue_time_us":20,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":6,"replication_time_us":1797198,"spinlock_wait_cycles":14080}]]}
I20260812 08:02:00.763242 12783 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59446 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3922, attempt_no=0}) took 1821 ms. Trace:
I20260812 08:02:00.763357 12783 rpcz_store.cc:276] 0812 08:01:58.941519 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:58.941571 (+    52us) service_pool.cc:224] Handling call
0812 08:02:00.763226 (+1821655us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:58.941884 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:58.942105 (+   221us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:58.942109 (+     4us) write_op.cc:435] Acquired schema lock
0812 08:01:58.942111 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:58.942217 (+   106us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:58.942223 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:58.942226 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:58.942285 (+    59us) tablet.cc:701] Row locks acquired
0812 08:01:58.942287 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:58.942318 (+    31us) write_op.cc:270] Start()
0812 08:01:58.942337 (+    19us) write_op.cc:276] Timestamp: P: 1786521718942235 usec, L: 0
0812 08:01:58.942339 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:58.942444 (+   105us) log.cc:844] Serialized 696 byte log entry
0812 08:02:00.736927 (+1794483us) op_driver.cc:464] REPLICATION: finished
0812 08:02:00.762716 (+ 25789us) write_op.cc:301] APPLY: starting
0812 08:02:00.762730 (+    14us) tablet.cc:1367] starting BulkCheckPresence
0812 08:02:00.762765 (+    35us) tablet.cc:1370] finished BulkCheckPresence
0812 08:02:00.762777 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:02:00.762951 (+   174us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:02:00.762979 (+    28us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:02:00.762983 (+     4us) write_op.cc:312] APPLY: finished
0812 08:02:00.763046 (+    63us) log.cc:844] Serialized 77 byte log entry
0812 08:02:00.763137 (+    91us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:02:00.763153 (+    16us) write_op.cc:454] Released schema lock
0812 08:02:00.763158 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":25734,"num_ops":10,"prepare.queue_time_us":43,"prepare.run_cpu_time_us":486,"prepare.run_wall_time_us":628,"raft.queue_time_us":19,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":5,"replication_time_us":1794575,"spinlock_wait_cycles":19456}]]}
I20260812 08:02:00.764324 12783 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59446 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3924, attempt_no=0}) took 1820 ms. Trace:
W20260812 08:02:00.764727  8145 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39074: Illegal state: replica 6801d45ca6614156906e5d571ebcb8a6 is not leader of this config: current role FOLLOWER
I20260812 08:02:00.764561 12783 rpcz_store.cc:276] 0812 08:01:58.943675 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:58.943712 (+    37us) service_pool.cc:224] Handling call
0812 08:02:00.764314 (+1820602us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:58.944449 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:58.944523 (+    74us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:58.944526 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:58.944528 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:58.944605 (+    77us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:58.944610 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:58.944613 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:58.944672 (+    59us) tablet.cc:701] Row locks acquired
0812 08:01:58.944674 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:58.944700 (+    26us) write_op.cc:270] Start()
0812 08:01:58.944717 (+    17us) write_op.cc:276] Timestamp: P: 1786521718944620 usec, L: 0
0812 08:01:58.944719 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:58.944804 (+    85us) log.cc:844] Serialized 686 byte log entry
0812 08:02:00.736976 (+1792172us) op_driver.cc:464] REPLICATION: finished
0812 08:02:00.763879 (+ 26903us) write_op.cc:301] APPLY: starting
0812 08:02:00.763889 (+    10us) tablet.cc:1367] starting BulkCheckPresence
0812 08:02:00.763920 (+    31us) tablet.cc:1370] finished BulkCheckPresence
0812 08:02:00.763931 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:02:00.764104 (+   173us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:02:00.764132 (+    28us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:02:00.764137 (+     5us) write_op.cc:312] APPLY: finished
0812 08:02:00.764194 (+    57us) log.cc:844] Serialized 77 byte log entry
0812 08:02:00.764240 (+    46us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:02:00.764254 (+    14us) write_op.cc:454] Released schema lock
0812 08:02:00.764259 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":26877,"mutex_wait_us":20,"num_ops":10,"prepare.queue_time_us":515,"prepare.run_cpu_time_us":414,"prepare.run_wall_time_us":414,"raft.queue_time_us":268,"raft.run_cpu_time_us":6,"raft.run_wall_time_us":7,"replication_time_us":1792248}]]}
I20260812 08:02:00.745671 12784 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59446 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3919, attempt_no=0}) took 1810 ms. Trace:
I20260812 08:02:00.746199 12790 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59446 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3920, attempt_no=0}) took 1809 ms. Trace:
W20260812 08:02:00.766057  7321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:40918: Illegal state: replica 16901c5559a748a9aa770d73e254099c is not leader of this config: current role FOLLOWER
I20260812 08:02:00.766006 12790 rpcz_store.cc:276] 0812 08:01:58.937184 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:58.937419 (+   235us) service_pool.cc:224] Handling call
0812 08:02:00.746190 (+1808771us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:58.938769 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:58.938840 (+    71us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:58.938844 (+     4us) write_op.cc:435] Acquired schema lock
0812 08:01:58.938845 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:58.938930 (+    85us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:58.938935 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:58.938937 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:58.938978 (+    41us) tablet.cc:701] Row locks acquired
0812 08:01:58.938979 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:58.939004 (+    25us) write_op.cc:270] Start()
0812 08:01:58.939019 (+    15us) write_op.cc:276] Timestamp: P: 1786521718938924 usec, L: 0
0812 08:01:58.939020 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:01:58.939122 (+   102us) log.cc:844] Serialized 696 byte log entry
0812 08:02:00.736778 (+1797656us) op_driver.cc:464] REPLICATION: finished
0812 08:02:00.745818 (+  9040us) write_op.cc:301] APPLY: starting
0812 08:02:00.745826 (+     8us) tablet.cc:1367] starting BulkCheckPresence
0812 08:02:00.745852 (+    26us) tablet.cc:1370] finished BulkCheckPresence
0812 08:02:00.745863 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:02:00.746015 (+   152us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:02:00.746039 (+    24us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:02:00.746043 (+     4us) write_op.cc:312] APPLY: finished
0812 08:02:00.746101 (+    58us) log.cc:844] Serialized 77 byte log entry
0812 08:02:00.746133 (+    32us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:02:00.746144 (+    11us) write_op.cc:454] Released schema lock
0812 08:02:00.746148 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":9014,"num_ops":10,"prepare.queue_time_us":1086,"prepare.run_cpu_time_us":431,"prepare.run_wall_time_us":585,"raft.queue_time_us":202,"raft.run_cpu_time_us":6,"raft.run_wall_time_us":6,"replication_time_us":1797744,"thread_start_us":46,"threads_started":1}]]}
I20260812 08:02:00.763583 12788 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:59446 (ReqId={client: 5bf9fd23bf7740e981a00aebd4969f65, seq_no=3923, attempt_no=0}) took 1819 ms. Trace:
W20260812 08:02:00.759469 10000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Illegal state: replica faafbc72cbd444ff90977a2cdb091b72 is not leader of this config: current role FOLLOWER
I20260812 08:02:00.766371 12788 rpcz_store.cc:276] 0812 08:01:58.943615 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:58.943676 (+    61us) service_pool.cc:224] Handling call
0812 08:02:00.763574 (+1819898us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:58.944003 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:58.944073 (+    70us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:58.944076 (+     3us) write_op.cc:435] Acquired schema lock
0812 08:01:58.944078 (+     2us) tablet.cc:662] Decoding operations
0812 08:01:58.944160 (+    82us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:58.944164 (+     4us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:58.944166 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:58.944211 (+    45us) tablet.cc:701] Row locks acquired
0812 08:01:58.944212 (+     1us) write_op.cc:260] PREPARE: finished
0812 08:01:58.944240 (+    28us) write_op.cc:270] Start()
0812 08:01:58.944258 (+    18us) write_op.cc:276] Timestamp: P: 1786521718944159 usec, L: 0
0812 08:01:58.944260 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:01:58.944358 (+    98us) log.cc:844] Serialized 696 byte log entry
0812 08:02:00.736954 (+1792596us) op_driver.cc:464] REPLICATION: finished
0812 08:02:00.763158 (+ 26204us) write_op.cc:301] APPLY: starting
0812 08:02:00.763169 (+    11us) tablet.cc:1367] starting BulkCheckPresence
0812 08:02:00.763199 (+    30us) tablet.cc:1370] finished BulkCheckPresence
0812 08:02:00.763211 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:02:00.763388 (+   177us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:02:00.763414 (+    26us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:02:00.763418 (+     4us) write_op.cc:312] APPLY: finished
0812 08:02:00.763469 (+    51us) log.cc:844] Serialized 77 byte log entry
0812 08:02:00.763509 (+    40us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:02:00.763522 (+    13us) write_op.cc:454] Released schema lock
0812 08:02:00.763526 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":26181,"num_ops":10,"prepare.queue_time_us":68,"prepare.run_cpu_time_us":429,"prepare.run_wall_time_us":430,"raft.queue_time_us":120,"raft.run_cpu_time_us":6,"raft.run_wall_time_us":7,"replication_time_us":1792685}]]}
I20260812 08:02:00.765710 12784 rpcz_store.cc:276] 0812 08:01:58.935253 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:01:58.935295 (+    42us) service_pool.cc:224] Handling call
0812 08:02:00.745659 (+1810364us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:01:58.936451 (+     0us) write_op.cc:183] PREPARE: starting on tablet f22cf4f6412248aa9e68c6d5d61153e8
0812 08:01:58.936544 (+    93us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:01:58.936548 (+     4us) write_op.cc:435] Acquired schema lock
0812 08:01:58.936549 (+     1us) tablet.cc:662] Decoding operations
0812 08:01:58.936625 (+    76us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:01:58.936640 (+    15us) write_op.cc:641] Partition lock acquired for write op
0812 08:01:58.936643 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 08:01:58.936701 (+    58us) tablet.cc:701] Row locks acquired
0812 08:01:58.936703 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:01:58.936739 (+    36us) write_op.cc:270] Start()
0812 08:01:58.936758 (+    19us) write_op.cc:276] Timestamp: P: 1786521718936652 usec, L: 0
0812 08:01:58.936759 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:01:58.938346 (+  1587us) log.cc:844] Serialized 686 byte log entry
0812 08:01:58.938455 (+   109us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x55b31c7d92d4. stack: 00007f9f3e4bfd35 00007f9f3e4c0325 00007f9f3dfcc82d 000055b3045c6b18 000055b3045c6b6b 000055b3045ca9f2 00007f9f4050c3bc 00007f9f40ac1367 00007f9f40ac04cc 00007f9f40abfd9a 00007f9f40ac572a 000055b30465644d 00007f9f3e4e3326 00007f9f3e4e3c4e 00007f9f3e4e539a 000055b30465644d
0812 08:02:00.736687 (+1798232us) op_driver.cc:464] REPLICATION: finished
0812 08:02:00.745286 (+  8599us) write_op.cc:301] APPLY: starting
0812 08:02:00.745296 (+    10us) tablet.cc:1367] starting BulkCheckPresence
0812 08:02:00.745324 (+    28us) tablet.cc:1370] finished BulkCheckPresence
0812 08:02:00.745325 (+     1us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:02:00.745487 (+   162us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:02:00.745511 (+    24us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:02:00.745516 (+     5us) write_op.cc:312] APPLY: finished
0812 08:02:00.745567 (+    51us) log.cc:844] Serialized 77 byte log entry
0812 08:02:00.745599 (+    32us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:02:00.745610 (+    11us) write_op.cc:454] Released schema lock
0812 08:02:00.745614 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":8559,"num_ops":10,"prepare.queue_time_us":913,"prepare.run_cpu_time_us":559,"prepare.run_wall_time_us":2304,"raft.queue_time_us":236,"raft.run_cpu_time_us":6,"raft.run_wall_time_us":6,"replication_time_us":1799912,"spinlock_wait_cycles":3425408,"thread_start_us":44,"threads_started":1}]]}
W20260812 08:02:00.767590  7317 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:40918: Illegal state: replica 16901c5559a748a9aa770d73e254099c is not leader of this config: current role FOLLOWER
W20260812 08:02:00.768018  7321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:40918: Illegal state: replica 16901c5559a748a9aa770d73e254099c is not leader of this config: current role FOLLOWER
W20260812 08:02:00.768558  7324 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:40918: Illegal state: replica 16901c5559a748a9aa770d73e254099c is not leader of this config: current role FOLLOWER
W20260812 08:02:00.769039  7324 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:40918: Illegal state: replica 16901c5559a748a9aa770d73e254099c is not leader of this config: current role FOLLOWER
W20260812 08:02:00.769181  7321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:40918: Illegal state: replica 16901c5559a748a9aa770d73e254099c is not leader of this config: current role FOLLOWER
W20260812 08:02:00.769868 10420 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:02:00.770200 10412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:02:00.770519 10420 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:02:00.773553 10420 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:02:00.773628 10412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:02:00.774122  7939 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35090: Illegal state: replica 780d7e64a8214418a371f35351b113df is not leader of this config: current role FOLLOWER
W20260812 08:02:00.774950  7939 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35090: Illegal state: replica 780d7e64a8214418a371f35351b113df is not leader of this config: current role FOLLOWER
W20260812 08:02:00.775051  7935 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35090: Illegal state: replica 780d7e64a8214418a371f35351b113df is not leader of this config: current role FOLLOWER
W20260812 08:02:00.776206  7935 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35090: Illegal state: replica 780d7e64a8214418a371f35351b113df is not leader of this config: current role FOLLOWER
W20260812 08:02:00.777446  7935 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35090: Illegal state: replica 780d7e64a8214418a371f35351b113df is not leader of this config: current role FOLLOWER
W20260812 08:02:00.777572  7943 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35090: Illegal state: replica 780d7e64a8214418a371f35351b113df is not leader of this config: current role FOLLOWER
W20260812 08:02:00.778007  7939 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35090: Illegal state: replica 780d7e64a8214418a371f35351b113df is not leader of this config: current role FOLLOWER
W20260812 08:02:00.780004 10412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34500: Illegal state: replica 3ae3f7a6e9f943e996a6dcea504878d0 is not leader of this config: current role FOLLOWER
W20260812 08:02:00.781307  8145 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39074: Illegal state: replica 6801d45ca6614156906e5d571ebcb8a6 is not leader of this config: current role FOLLOWER
W20260812 08:02:00.782094  8145 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39074: Illegal state: replica 6801d45ca6614156906e5d571ebcb8a6 is not leader of this config: current role FOLLOWER
W20260812 08:02:00.783432  8145 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39074: Illegal state: replica 6801d45ca6614156906e5d571ebcb8a6 is not leader of this config: current role FOLLOWER
W20260812 08:02:00.783479  8158 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39074: Illegal state: replica 6801d45ca6614156906e5d571ebcb8a6 is not leader of this config: current role FOLLOWER
W20260812 08:02:00.784364 10000 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59446: Illegal state: replica faafbc72cbd444ff90977a2cdb091b72 is not leader of this config: current role FOLLOWER
I20260812 08:02:00.791055  8003 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 780d7e64a8214418a371f35351b113df [term 27 FOLLOWER]: Refusing update from remote peer 1664e32d3cd0473592085aa522379f22: Log matching property violated. Preceding OpId in replica: term: 26 index: 3872. Preceding OpId from leader: term: 27 index: 3947. (index mismatch)
I20260812 08:02:00.791268 10065 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P faafbc72cbd444ff90977a2cdb091b72 [term 27 FOLLOWER]: Refusing update from remote peer 1664e32d3cd0473592085aa522379f22: Log matching property violated. Preceding OpId in replica: term: 26 index: 3945. Preceding OpId from leader: term: 27 index: 3947. (index mismatch)
I20260812 08:02:00.791772 12074 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 56b60424ed9f4a079689ad82a318d29a [term 27 FOLLOWER]: Refusing update from remote peer 1664e32d3cd0473592085aa522379f22: Log matching property violated. Preceding OpId in replica: term: 26 index: 3945. Preceding OpId from leader: term: 27 index: 3947. (index mismatch)
I20260812 08:02:00.791909  8225 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 26 FOLLOWER]: Advancing to term 27
I20260812 08:02:00.792142 12745 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "780d7e64a8214418a371f35351b113df" member_type: VOTER last_known_addr { host: "127.5.142.134" port: 33021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3946, Last known committed idx: 3870, Time since last communication: 0.000s
I20260812 08:02:00.793336 12745 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "faafbc72cbd444ff90977a2cdb091b72" member_type: VOTER last_known_addr { host: "127.5.142.132" port: 43153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3946, Last known committed idx: 3945, Time since last communication: 0.000s
I20260812 08:02:00.793754  8225 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 6801d45ca6614156906e5d571ebcb8a6 [term 27 FOLLOWER]: Refusing update from remote peer 1664e32d3cd0473592085aa522379f22: Log matching property violated. Preceding OpId in replica: term: 26 index: 3945. Preceding OpId from leader: term: 27 index: 3947. (index mismatch)
I20260812 08:02:00.795171 10476 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 3ae3f7a6e9f943e996a6dcea504878d0 [term 27 FOLLOWER]: Refusing update from remote peer 1664e32d3cd0473592085aa522379f22: Log matching property violated. Preceding OpId in replica: term: 26 index: 3945. Preceding OpId from leader: term: 27 index: 3947. (index mismatch)
I20260812 08:02:00.795732  7378 raft_consensus.cc:3060] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 26 FOLLOWER]: Advancing to term 27
I20260812 08:02:00.796929  7378 raft_consensus.cc:1275] T f22cf4f6412248aa9e68c6d5d61153e8 P 16901c5559a748a9aa770d73e254099c [term 27 FOLLOWER]: Refusing update from remote peer 1664e32d3cd0473592085aa522379f22: Log matching property violated. Preceding OpId in replica: term: 26 index: 3945. Preceding OpId from leader: term: 27 index: 3947. (index mismatch)
I20260812 08:02:00.799036 12818 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56b60424ed9f4a079689ad82a318d29a" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 38363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3946, Last known committed idx: 3937, Time since last communication: 0.000s
I20260812 08:02:00.799424 12822 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae3f7a6e9f943e996a6dcea504878d0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3946, Last known committed idx: 3937, Time since last communication: 0.000s
I20260812 08:02:00.799569 12820 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6801d45ca6614156906e5d571ebcb8a6" member_type: VOTER last_known_addr { host: "127.5.142.135" port: 42409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3946, Last known committed idx: 3945, Time since last communication: 0.000s
I20260812 08:02:00.799923 12821 consensus_queue.cc:1048] T f22cf4f6412248aa9e68c6d5d61153e8 P 1664e32d3cd0473592085aa522379f22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16901c5559a748a9aa770d73e254099c" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3946, Last known committed idx: 3937, Time since last communication: 0.000s
I20260812 08:02:00.873647 12827 mvcc.cc:204] Tried to move back new op lower bound from 7317592968361467904 to 7317592967785480192. Current Snapshot: MvccSnapshot[applied={T|T < 7317592960636633088 or (T in {7317592960655552512,7317592960659677184,7317592960661708800,7317592960664768512,7317592960691924992,7317592960698499072,7317592960700567552,7317592960721498112,7317592960723390464,7317592960725159936,7317592960750731264,7317592960757714944,7317592960764526592,7317592960773832704,7317592960787394560,7317592960795275264,7317592960797163520,7317592968354873344})}]
I20260812 08:02:01.195909  7356 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 08:02:01.199759  8192 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 08:02:01.201848 12406 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260812 08:02:01.204809 10450 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 08:02:01.205317  7975 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 08:02:01.205618 10043 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 08:02:01.206019 12041 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |       Address       | Status
----------------------------------+---------------------+---------
 23389c6fd7634cbca868c16d3f1c61d2 | 127.5.142.190:44379 | HEALTHY

Unusual flags for Master:
               Flag               |                                                                                          Value                                                                                           |      Tags       |         Master
----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
 ipki_ca_key_size                 | 768                                                                                                                                                                                      | experimental    | all 1 server(s) checked
 ipki_server_key_size             | 768                                                                                                                                                                                      | experimental    | all 1 server(s) checked
 never_fsync                      | true                                                                                                                                                                                     | unsafe,advanced | all 1 server(s) checked
 openssl_security_level_override  | 0                                                                                                                                                                                        | unsafe,hidden   | all 1 server(s) checked
 rpc_reuseport                    | true                                                                                                                                                                                     | experimental    | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports | true                                                                                                                                                                                     | unsafe          | all 1 server(s) checked
 server_dump_info_format          | pb                                                                                                                                                                                       | hidden          | all 1 server(s) checked
 server_dump_info_path            | /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden          | all 1 server(s) checked
 tsk_num_rsa_bits                 | 512                                                                                                                                                                                      | experimental    | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |        Value        |         Master
---------------------+---------------------+-------------------------
 builtin_ntp_servers | 127.5.142.148:43055 | all 1 server(s) checked
 time_source         | builtin             | all 1 server(s) checked

Tablet Server Summary
               UUID               |       Address       | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
 1664e32d3cd0473592085aa522379f22 | 127.5.142.133:33105 | HEALTHY | <none>   |       1        |       0
 16901c5559a748a9aa770d73e254099c | 127.5.142.131:33183 | HEALTHY | <none>   |       0        |       0
 3ae3f7a6e9f943e996a6dcea504878d0 | 127.5.142.129:41863 | HEALTHY | <none>   |       0        |       0
 56b60424ed9f4a079689ad82a318d29a | 127.5.142.130:38363 | HEALTHY | <none>   |       0        |       0
 6801d45ca6614156906e5d571ebcb8a6 | 127.5.142.135:42409 | HEALTHY | <none>   |       0        |       0
 780d7e64a8214418a371f35351b113df | 127.5.142.134:33021 | HEALTHY | <none>   |       0        |       0
 faafbc72cbd444ff90977a2cdb091b72 | 127.5.142.132:43153 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       7

Unusual flags for Tablet Server:
               Flag               |                                                                                        Value                                                                                         |      Tags       |      Tablet Server
----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
 ipki_server_key_size             | 768                                                                                                                                                                                  | experimental    | all 7 server(s) checked
 local_ip_for_outbound_sockets    | 127.5.142.129                                                                                                                                                                        | experimental    | 127.5.142.129:41863
 local_ip_for_outbound_sockets    | 127.5.142.130                                                                                                                                                                        | experimental    | 127.5.142.130:38363
 local_ip_for_outbound_sockets    | 127.5.142.131                                                                                                                                                                        | experimental    | 127.5.142.131:33183
 local_ip_for_outbound_sockets    | 127.5.142.132                                                                                                                                                                        | experimental    | 127.5.142.132:43153
 local_ip_for_outbound_sockets    | 127.5.142.133                                                                                                                                                                        | experimental    | 127.5.142.133:33105
 local_ip_for_outbound_sockets    | 127.5.142.134                                                                                                                                                                        | experimental    | 127.5.142.134:33021
 local_ip_for_outbound_sockets    | 127.5.142.135                                                                                                                                                                        | experimental    | 127.5.142.135:42409
 never_fsync                      | true                                                                                                                                                                                 | unsafe,advanced | all 7 server(s) checked
 openssl_security_level_override  | 0                                                                                                                                                                                    | unsafe,hidden   | all 7 server(s) checked
 rpc_server_allow_ephemeral_ports | true                                                                                                                                                                                 | unsafe          | all 7 server(s) checked
 server_dump_info_format          | pb                                                                                                                                                                                   | hidden          | all 7 server(s) checked
 server_dump_info_path            | /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden          | 127.5.142.129:41863
 server_dump_info_path            | /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden          | 127.5.142.130:38363
 server_dump_info_path            | /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden          | 127.5.142.131:33183
 server_dump_info_path            | /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden          | 127.5.142.132:43153
 server_dump_info_path            | /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden          | 127.5.142.133:33105
 server_dump_info_path            | /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden          | 127.5.142.134:33021
 server_dump_info_path            | /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden          | 127.5.142.135:42409

Flags of checked categories for Tablet Server:
        Flag         |        Value        |      Tablet Server
---------------------+---------------------+-------------------------
 builtin_ntp_servers | 127.5.142.148:43055 | all 7 server(s) checked
 time_source         | builtin             | all 7 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 8 server(s) checked

Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
   Name    | RF | Status  | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-----------+----+---------+---------------+---------+------------+------------------+-------------
 TestTable | 7  | HEALTHY | 1             | 1       | 0          | 0                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 7
 Tables         | 1
 Tablets        | 1
 Replicas       | 7

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

OK
I20260812 08:02:01.252219  5690 log_verifier.cc:126] Checking tablet f22cf4f6412248aa9e68c6d5d61153e8
I20260812 08:02:03.221266  5690 log_verifier.cc:177] Verified matching terms for 4021 ops in tablet f22cf4f6412248aa9e68c6d5d61153e8
I20260812 08:02:03.602892  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 10270
I20260812 08:02:03.626683  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 11937
I20260812 08:02:03.644469  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 7258
I20260812 08:02:03.660741  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 9944
I20260812 08:02:03.677482  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 12288
I20260812 08:02:03.695366  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 7878
I20260812 08:02:03.709635  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 8090
I20260812 08:02:03.741806  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 6703
2026-08-12T08:02:03Z chronyd exiting
[       OK ] RaftConsensusITest.MultiThreadedInsertWithFailovers (70053 ms)
[ RUN      ] RaftConsensusITest.TestReplaceChangeConfigOperation
I20260812 08:02:03.783046  5690 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:02:03.783097  5690 ts_itest-base.cc:116] --------------
I20260812 08:02:03.783113  5690 ts_itest-base.cc:117] 3 tablet servers
I20260812 08:02:03.783133  5690 ts_itest-base.cc:118] 3 replicas per TS
I20260812 08:02:03.783149  5690 ts_itest-base.cc:119] --------------
2026-08-12T08:02:03Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:02:03Z Disabled control of system clock
I20260812 08:02:03.793347  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.142.190:41149
--webserver_interface=127.5.142.190
--webserver_port=0
--builtin_ntp_servers=127.5.142.148:34913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.142.190:41149
--master_add_server_when_underreplicated=false
--catalog_manager_wait_for_new_tablets_to_elect_leader=false with env {}
W20260812 08:02:03.900381 12947 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:03.900594 12947 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:03.900647 12947 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:03.903924 12947 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:02:03.903981 12947 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:03.904008 12947 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:02:03.904031 12947 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:02:03.907809 12947 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:34913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
--catalog_manager_wait_for_new_tablets_to_elect_leader=false
--master_add_server_when_underreplicated=false
--ipki_ca_key_size=768
--master_addresses=127.5.142.190:41149
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.142.190:41149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.5.142.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.12947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:03.908877 12947 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:03.910025 12947 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:03.915249 12960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:03.915374 12954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:03.915468 12947 server_base.cc:1061] running on GCE node
W20260812 08:02:03.915256 12953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:03.916020 12947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:03.916953 12947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:03.918110 12947 hybrid_clock.cc:648] HybridClock initialized: now 1786521723918084 us; error 40 us; skew 500 ppm
I20260812 08:02:03.919785 12947 webserver.cc:533] Webserver started at http://127.5.142.190:45453/ using document root <none> and password file <none>
I20260812 08:02:03.920331 12947 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:03.920385 12947 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:03.920567 12947 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:03.922174 12947 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "d8011dd427634b0ab68b77d80ab46f2f"
format_stamp: "Formatted at 2026-08-12 08:02:03 on dist-test-slave-ncp9"
I20260812 08:02:03.922639 12947 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "d8011dd427634b0ab68b77d80ab46f2f"
format_stamp: "Formatted at 2026-08-12 08:02:03 on dist-test-slave-ncp9"
I20260812 08:02:03.925865 12947 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.000s	sys 0.004s
I20260812 08:02:03.927882 12971 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:03.928848 12947 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260812 08:02:03.928987 12947 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
uuid: "d8011dd427634b0ab68b77d80ab46f2f"
format_stamp: "Formatted at 2026-08-12 08:02:03 on dist-test-slave-ncp9"
I20260812 08:02:03.929076 12947 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:03.951537 12947 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:03.952313 12947 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:03.952503 12947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:03.960114 13068 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.190:41149 every 8 connection(s)
I20260812 08:02:03.960126 12947 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.190:41149
I20260812 08:02:03.961246 12947 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:02:03.964155 13069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:03.969115  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 12947
I20260812 08:02:03.969225  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:02:03.969311 13069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8011dd427634b0ab68b77d80ab46f2f: Bootstrap starting.
I20260812 08:02:03.971850 13069 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8011dd427634b0ab68b77d80ab46f2f: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:03.972715 13069 log.cc:826] T 00000000000000000000000000000000 P d8011dd427634b0ab68b77d80ab46f2f: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:03.974720 13069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8011dd427634b0ab68b77d80ab46f2f: No bootstrap required, opened a new log
I20260812 08:02:03.977638 13069 raft_consensus.cc:359] T 00000000000000000000000000000000 P d8011dd427634b0ab68b77d80ab46f2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8011dd427634b0ab68b77d80ab46f2f" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 41149 } }
I20260812 08:02:03.977866 13069 raft_consensus.cc:385] T 00000000000000000000000000000000 P d8011dd427634b0ab68b77d80ab46f2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:03.977927 13069 raft_consensus.cc:740] T 00000000000000000000000000000000 P d8011dd427634b0ab68b77d80ab46f2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8011dd427634b0ab68b77d80ab46f2f, State: Initialized, Role: FOLLOWER
I20260812 08:02:03.978372 13069 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8011dd427634b0ab68b77d80ab46f2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8011dd427634b0ab68b77d80ab46f2f" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 41149 } }
I20260812 08:02:03.978524 13069 raft_consensus.cc:399] T 00000000000000000000000000000000 P d8011dd427634b0ab68b77d80ab46f2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:03.978619 13069 raft_consensus.cc:493] T 00000000000000000000000000000000 P d8011dd427634b0ab68b77d80ab46f2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:03.978744 13069 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d8011dd427634b0ab68b77d80ab46f2f [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:03.979475 13069 raft_consensus.cc:515] T 00000000000000000000000000000000 P d8011dd427634b0ab68b77d80ab46f2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8011dd427634b0ab68b77d80ab46f2f" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 41149 } }
I20260812 08:02:03.979790 13069 leader_election.cc:304] T 00000000000000000000000000000000 P d8011dd427634b0ab68b77d80ab46f2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8011dd427634b0ab68b77d80ab46f2f; no voters: 
I20260812 08:02:03.980037 13069 leader_election.cc:290] T 00000000000000000000000000000000 P d8011dd427634b0ab68b77d80ab46f2f [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:02:03.980156 13075 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d8011dd427634b0ab68b77d80ab46f2f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:02:03.980370 13075 raft_consensus.cc:697] T 00000000000000000000000000000000 P d8011dd427634b0ab68b77d80ab46f2f [term 1 LEADER]: Becoming Leader. State: Replica: d8011dd427634b0ab68b77d80ab46f2f, State: Running, Role: LEADER
I20260812 08:02:03.980728 13075 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8011dd427634b0ab68b77d80ab46f2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8011dd427634b0ab68b77d80ab46f2f" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 41149 } }
I20260812 08:02:03.981091 13069 sys_catalog.cc:565] T 00000000000000000000000000000000 P d8011dd427634b0ab68b77d80ab46f2f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:02:03.982323 13076 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8011dd427634b0ab68b77d80ab46f2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8011dd427634b0ab68b77d80ab46f2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8011dd427634b0ab68b77d80ab46f2f" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 41149 } } }
I20260812 08:02:03.982443 13076 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8011dd427634b0ab68b77d80ab46f2f [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:03.982825 13085 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:02:03.983336 13077 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8011dd427634b0ab68b77d80ab46f2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8011dd427634b0ab68b77d80ab46f2f. Latest consensus state: current_term: 1 leader_uuid: "d8011dd427634b0ab68b77d80ab46f2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8011dd427634b0ab68b77d80ab46f2f" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 41149 } } }
I20260812 08:02:03.983486 13077 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8011dd427634b0ab68b77d80ab46f2f [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:03.985962 13085 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:02:03.991384 13085 catalog_manager.cc:1383] Generated new cluster ID: b48916b208b44099b142b4139aef8fb3
I20260812 08:02:03.991578 13085 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:02:04.017179 13085 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:02:04.018386 13085 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:02:04.037776 13085 catalog_manager.cc:6092] T 00000000000000000000000000000000 P d8011dd427634b0ab68b77d80ab46f2f: Generated new TSK 0
I20260812 08:02:04.038477 13085 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:02:04.043929  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.129:0
--local_ip_for_outbound_sockets=127.5.142.129
--webserver_interface=127.5.142.129
--webserver_port=0
--tserver_master_addrs=127.5.142.190:41149
--builtin_ntp_servers=127.5.142.148:34913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false with env {}
W20260812 08:02:04.154646 13103 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 08:02:04.154892 13103 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:04.154913 13103 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:04.154960 13103 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:04.158375 13103 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:04.158499 13103 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.129
I20260812 08:02:04.162787 13103 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:34913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.5.142.129
--webserver_port=0
--tserver_master_addrs=127.5.142.190:41149
--never_fsync=true
--heap_profile_path=/tmp/kudu.13103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.129

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:04.163945 13103 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:04.165206 13103 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:04.171742 13119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:04.171873 13120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:04.171748 13125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:04.172055 13103 server_base.cc:1061] running on GCE node
I20260812 08:02:04.172410 13103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:04.172979 13103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:04.174154 13103 hybrid_clock.cc:648] HybridClock initialized: now 1786521724174136 us; error 33 us; skew 500 ppm
I20260812 08:02:04.175952 13103 webserver.cc:533] Webserver started at http://127.5.142.129:39707/ using document root <none> and password file <none>
I20260812 08:02:04.176554 13103 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:04.176620 13103 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:04.176818 13103 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:04.178421 13103 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "fa6eccb1153546f08fec284993816ab5"
format_stamp: "Formatted at 2026-08-12 08:02:04 on dist-test-slave-ncp9"
I20260812 08:02:04.178884 13103 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "fa6eccb1153546f08fec284993816ab5"
format_stamp: "Formatted at 2026-08-12 08:02:04 on dist-test-slave-ncp9"
I20260812 08:02:04.182210 13103 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:02:04.184513 13134 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:04.185612 13103 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 08:02:04.185726 13103 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "fa6eccb1153546f08fec284993816ab5"
format_stamp: "Formatted at 2026-08-12 08:02:04 on dist-test-slave-ncp9"
I20260812 08:02:04.185817 13103 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:04.201895 13103 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:04.202692 13103 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:04.202893 13103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:04.203540 13103 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:04.204697 13103 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:04.204751 13103 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:04.204807 13103 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:04.204836 13103 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:04.213897 13103 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.129:45937
I20260812 08:02:04.213928 13312 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.129:45937 every 8 connection(s)
I20260812 08:02:04.214879 13103 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:02:04.221530  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 13103
I20260812 08:02:04.221637  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:02:04.224077 13314 heartbeater.cc:344] Connected to a master server at 127.5.142.190:41149
I20260812 08:02:04.224150  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.130:0
--local_ip_for_outbound_sockets=127.5.142.130
--webserver_interface=127.5.142.130
--webserver_port=0
--tserver_master_addrs=127.5.142.190:41149
--builtin_ntp_servers=127.5.142.148:34913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false with env {}
I20260812 08:02:04.224342 13314 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:04.224852 13314 heartbeater.cc:507] Master 127.5.142.190:41149 requested a full tablet report, sending...
I20260812 08:02:04.226307 13000 ts_manager.cc:194] Registered new tserver with Master: fa6eccb1153546f08fec284993816ab5 (127.5.142.129:45937)
I20260812 08:02:04.227421 13000 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.129:38151
W20260812 08:02:04.332365 13319 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 08:02:04.332564 13319 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:04.332584 13319 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:04.332630 13319 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:04.335891 13319 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:04.335994 13319 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.130
I20260812 08:02:04.339689 13319 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:34913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.5.142.130
--webserver_port=0
--tserver_master_addrs=127.5.142.190:41149
--never_fsync=true
--heap_profile_path=/tmp/kudu.13319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.130

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:04.340770 13319 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:04.341892 13319 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:04.347975 13336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:04.347988 13332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:04.348101 13331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:04.348325 13319 server_base.cc:1061] running on GCE node
I20260812 08:02:04.348644 13319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:04.349216 13319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:04.350365 13319 hybrid_clock.cc:648] HybridClock initialized: now 1786521724350351 us; error 32 us; skew 500 ppm
I20260812 08:02:04.352041 13319 webserver.cc:533] Webserver started at http://127.5.142.130:40739/ using document root <none> and password file <none>
I20260812 08:02:04.352595 13319 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:04.352656 13319 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:04.352864 13319 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:04.354431 13319 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "d9fa5cb6c5034f9290343a342e360622"
format_stamp: "Formatted at 2026-08-12 08:02:04 on dist-test-slave-ncp9"
I20260812 08:02:04.354892 13319 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "d9fa5cb6c5034f9290343a342e360622"
format_stamp: "Formatted at 2026-08-12 08:02:04 on dist-test-slave-ncp9"
I20260812 08:02:04.358309 13319 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.001s
I20260812 08:02:04.360545 13352 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:04.361631 13319 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 08:02:04.361753 13319 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "d9fa5cb6c5034f9290343a342e360622"
format_stamp: "Formatted at 2026-08-12 08:02:04 on dist-test-slave-ncp9"
I20260812 08:02:04.361845 13319 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:04.378969 13319 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:04.379676 13319 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:04.379854 13319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:04.380445 13319 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:04.381594 13319 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:04.381656 13319 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:04.381743 13319 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:04.381773 13319 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:04.390942 13319 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.130:42417
I20260812 08:02:04.390980 13540 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.130:42417 every 8 connection(s)
I20260812 08:02:04.391930 13319 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:02:04.400094 13541 heartbeater.cc:344] Connected to a master server at 127.5.142.190:41149
I20260812 08:02:04.400331 13541 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:04.400835 13541 heartbeater.cc:507] Master 127.5.142.190:41149 requested a full tablet report, sending...
I20260812 08:02:04.401400  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 13319
I20260812 08:02:04.401503  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:02:04.401988 13000 ts_manager.cc:194] Registered new tserver with Master: d9fa5cb6c5034f9290343a342e360622 (127.5.142.130:42417)
I20260812 08:02:04.402587 13000 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.130:40561
I20260812 08:02:04.403942  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.131:0
--local_ip_for_outbound_sockets=127.5.142.131
--webserver_interface=127.5.142.131
--webserver_port=0
--tserver_master_addrs=127.5.142.190:41149
--builtin_ntp_servers=127.5.142.148:34913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false with env {}
W20260812 08:02:04.514846 13544 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 08:02:04.515136 13544 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:04.515177 13544 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:04.515226 13544 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:04.518501 13544 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:04.518611 13544 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.131
I20260812 08:02:04.522344 13544 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:34913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.5.142.131
--webserver_port=0
--tserver_master_addrs=127.5.142.190:41149
--never_fsync=true
--heap_profile_path=/tmp/kudu.13544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.131

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:04.523378 13544 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:04.524464 13544 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:04.530867 13556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:04.530982 13544 server_base.cc:1061] running on GCE node
W20260812 08:02:04.530872 13559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:04.530869 13555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:04.531491 13544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:04.532037 13544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:04.533195 13544 hybrid_clock.cc:648] HybridClock initialized: now 1786521724533176 us; error 30 us; skew 500 ppm
I20260812 08:02:04.534888 13544 webserver.cc:533] Webserver started at http://127.5.142.131:46297/ using document root <none> and password file <none>
I20260812 08:02:04.535465 13544 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:04.535524 13544 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:04.535714 13544 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:04.537392 13544 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "9dfa38318eab49ecb69a57dc81508726"
format_stamp: "Formatted at 2026-08-12 08:02:04 on dist-test-slave-ncp9"
I20260812 08:02:04.537943 13544 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "9dfa38318eab49ecb69a57dc81508726"
format_stamp: "Formatted at 2026-08-12 08:02:04 on dist-test-slave-ncp9"
I20260812 08:02:04.541507 13544 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.001s
I20260812 08:02:04.543608 13571 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:04.544574 13544 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:04.544678 13544 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "9dfa38318eab49ecb69a57dc81508726"
format_stamp: "Formatted at 2026-08-12 08:02:04 on dist-test-slave-ncp9"
I20260812 08:02:04.544762 13544 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:04.570536 13544 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:04.571235 13544 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:04.571419 13544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:04.572032 13544 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:04.573179 13544 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:04.573233 13544 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:04.573294 13544 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:04.573319 13544 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:04.582777 13544 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.131:33535
I20260812 08:02:04.582830 13749 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.131:33535 every 8 connection(s)
I20260812 08:02:04.583876 13544 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:02:04.591346  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 13544
I20260812 08:02:04.591465  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:02:04.592631 13750 heartbeater.cc:344] Connected to a master server at 127.5.142.190:41149
I20260812 08:02:04.592877 13750 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:04.593521 13750 heartbeater.cc:507] Master 127.5.142.190:41149 requested a full tablet report, sending...
I20260812 08:02:04.594569 12995 ts_manager.cc:194] Registered new tserver with Master: 9dfa38318eab49ecb69a57dc81508726 (127.5.142.131:33535)
I20260812 08:02:04.595136 12995 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.131:38621
I20260812 08:02:04.605592  5690 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:02:04.614279 12995 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34474:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
W20260812 08:02:04.616688 12995 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 08:02:04.633070 13437 tablet_service.cc:1511] Processing CreateTablet for tablet 625690021b4c4ccf96b681a7bea2e8a8 (DEFAULT_TABLE table=TestTable [id=2b7a3bcd281341529b1920b5091f6ee7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:04.633070 13646 tablet_service.cc:1511] Processing CreateTablet for tablet 625690021b4c4ccf96b681a7bea2e8a8 (DEFAULT_TABLE table=TestTable [id=2b7a3bcd281341529b1920b5091f6ee7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:04.633070 13207 tablet_service.cc:1511] Processing CreateTablet for tablet 625690021b4c4ccf96b681a7bea2e8a8 (DEFAULT_TABLE table=TestTable [id=2b7a3bcd281341529b1920b5091f6ee7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:04.634011 13437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 625690021b4c4ccf96b681a7bea2e8a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:04.634021 13207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 625690021b4c4ccf96b681a7bea2e8a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:04.634021 13646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 625690021b4c4ccf96b681a7bea2e8a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:04.638906 13776 tablet_bootstrap.cc:492] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622: Bootstrap starting.
I20260812 08:02:04.640106 13780 tablet_bootstrap.cc:492] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726: Bootstrap starting.
I20260812 08:02:04.641634 13776 tablet_bootstrap.cc:654] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:04.642285 13780 tablet_bootstrap.cc:654] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:04.642522 13776 log.cc:826] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:04.643100 13780 log.cc:826] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:04.643256 13784 tablet_bootstrap.cc:492] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5: Bootstrap starting.
I20260812 08:02:04.644232 13776 tablet_bootstrap.cc:492] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622: No bootstrap required, opened a new log
I20260812 08:02:04.644419 13776 ts_tablet_manager.cc:1403] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622: Time spent bootstrapping tablet: real 0.006s	user 0.004s	sys 0.000s
I20260812 08:02:04.644812 13780 tablet_bootstrap.cc:492] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726: No bootstrap required, opened a new log
I20260812 08:02:04.645021 13780 ts_tablet_manager.cc:1403] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726: Time spent bootstrapping tablet: real 0.005s	user 0.001s	sys 0.004s
I20260812 08:02:04.645320 13784 tablet_bootstrap.cc:654] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:04.646080 13784 log.cc:826] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:04.647243 13776 raft_consensus.cc:359] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfa38318eab49ecb69a57dc81508726" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33535 } } peers { permanent_uuid: "d9fa5cb6c5034f9290343a342e360622" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 42417 } } peers { permanent_uuid: "fa6eccb1153546f08fec284993816ab5" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 45937 } }
I20260812 08:02:04.647435 13776 raft_consensus.cc:740] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9fa5cb6c5034f9290343a342e360622, State: Initialized, Role: FOLLOWER
I20260812 08:02:04.647729 13776 consensus_queue.cc:260] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfa38318eab49ecb69a57dc81508726" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33535 } } peers { permanent_uuid: "d9fa5cb6c5034f9290343a342e360622" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 42417 } } peers { permanent_uuid: "fa6eccb1153546f08fec284993816ab5" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 45937 } }
I20260812 08:02:04.647868 13784 tablet_bootstrap.cc:492] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5: No bootstrap required, opened a new log
I20260812 08:02:04.647891 13780 raft_consensus.cc:359] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfa38318eab49ecb69a57dc81508726" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33535 } } peers { permanent_uuid: "d9fa5cb6c5034f9290343a342e360622" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 42417 } } peers { permanent_uuid: "fa6eccb1153546f08fec284993816ab5" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 45937 } }
I20260812 08:02:04.648067 13784 ts_tablet_manager.cc:1403] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5: Time spent bootstrapping tablet: real 0.005s	user 0.003s	sys 0.000s
I20260812 08:02:04.648104 13780 raft_consensus.cc:740] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dfa38318eab49ecb69a57dc81508726, State: Initialized, Role: FOLLOWER
I20260812 08:02:04.648398 13780 consensus_queue.cc:260] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfa38318eab49ecb69a57dc81508726" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33535 } } peers { permanent_uuid: "d9fa5cb6c5034f9290343a342e360622" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 42417 } } peers { permanent_uuid: "fa6eccb1153546f08fec284993816ab5" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 45937 } }
I20260812 08:02:04.648554 13541 heartbeater.cc:499] Master 127.5.142.190:41149 was elected leader, sending a full tablet report...
I20260812 08:02:04.648578 13776 ts_tablet_manager.cc:1434] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:02:04.649184 13750 heartbeater.cc:499] Master 127.5.142.190:41149 was elected leader, sending a full tablet report...
I20260812 08:02:04.649245 13780 ts_tablet_manager.cc:1434] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726: Time spent starting tablet: real 0.004s	user 0.003s	sys 0.001s
I20260812 08:02:04.651333 13784 raft_consensus.cc:359] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfa38318eab49ecb69a57dc81508726" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33535 } } peers { permanent_uuid: "d9fa5cb6c5034f9290343a342e360622" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 42417 } } peers { permanent_uuid: "fa6eccb1153546f08fec284993816ab5" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 45937 } }
I20260812 08:02:04.651499 13784 raft_consensus.cc:740] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa6eccb1153546f08fec284993816ab5, State: Initialized, Role: FOLLOWER
I20260812 08:02:04.651742 13784 consensus_queue.cc:260] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfa38318eab49ecb69a57dc81508726" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33535 } } peers { permanent_uuid: "d9fa5cb6c5034f9290343a342e360622" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 42417 } } peers { permanent_uuid: "fa6eccb1153546f08fec284993816ab5" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 45937 } }
I20260812 08:02:04.652344 13314 heartbeater.cc:499] Master 127.5.142.190:41149 was elected leader, sending a full tablet report...
I20260812 08:02:04.652685 13784 ts_tablet_manager.cc:1434] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260812 08:02:04.659040  5690 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:02:04.660360  5690 ts_itest-base.cc:246] Waiting for 1 tablets on tserver fa6eccb1153546f08fec284993816ab5 to finish bootstrapping
I20260812 08:02:04.665733  5690 ts_itest-base.cc:246] Waiting for 1 tablets on tserver d9fa5cb6c5034f9290343a342e360622 to finish bootstrapping
I20260812 08:02:04.671119  5690 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 9dfa38318eab49ecb69a57dc81508726 to finish bootstrapping
I20260812 08:02:04.679641 13243 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "625690021b4c4ccf96b681a7bea2e8a8"
dest_uuid: "fa6eccb1153546f08fec284993816ab5"
 from {username='slave'} at 127.0.0.1:51960
I20260812 08:02:04.679872 13243 raft_consensus.cc:493] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 08:02:04.679965 13243 raft_consensus.cc:3060] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:04.680917 13243 raft_consensus.cc:515] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfa38318eab49ecb69a57dc81508726" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33535 } } peers { permanent_uuid: "d9fa5cb6c5034f9290343a342e360622" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 42417 } } peers { permanent_uuid: "fa6eccb1153546f08fec284993816ab5" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 45937 } }
I20260812 08:02:04.681993 13243 leader_election.cc:290] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5 [CANDIDATE]: Term 1 election: Requested vote from peers 9dfa38318eab49ecb69a57dc81508726 (127.5.142.131:33535), d9fa5cb6c5034f9290343a342e360622 (127.5.142.130:42417)
I20260812 08:02:04.686431 13466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625690021b4c4ccf96b681a7bea2e8a8" candidate_uuid: "fa6eccb1153546f08fec284993816ab5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d9fa5cb6c5034f9290343a342e360622"
I20260812 08:02:04.686661 13466 raft_consensus.cc:3060] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:04.686725 13683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625690021b4c4ccf96b681a7bea2e8a8" candidate_uuid: "fa6eccb1153546f08fec284993816ab5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9dfa38318eab49ecb69a57dc81508726"
I20260812 08:02:04.686946 13683 raft_consensus.cc:3060] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:04.687772 13466 raft_consensus.cc:2468] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa6eccb1153546f08fec284993816ab5 in term 1.
I20260812 08:02:04.687937 13683 raft_consensus.cc:2468] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa6eccb1153546f08fec284993816ab5 in term 1.
I20260812 08:02:04.688277 13140 leader_election.cc:304] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d9fa5cb6c5034f9290343a342e360622, fa6eccb1153546f08fec284993816ab5; no voters: 
I20260812 08:02:04.688592 13792 raft_consensus.cc:2804] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:02:04.688802 13792 raft_consensus.cc:697] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5 [term 1 LEADER]: Becoming Leader. State: Replica: fa6eccb1153546f08fec284993816ab5, State: Running, Role: LEADER
I20260812 08:02:04.689190 13792 consensus_queue.cc:237] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfa38318eab49ecb69a57dc81508726" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33535 } } peers { permanent_uuid: "d9fa5cb6c5034f9290343a342e360622" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 42417 } } peers { permanent_uuid: "fa6eccb1153546f08fec284993816ab5" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 45937 } }
I20260812 08:02:04.692711 13000 catalog_manager.cc:5719] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa6eccb1153546f08fec284993816ab5 (127.5.142.129). New cstate: current_term: 1 leader_uuid: "fa6eccb1153546f08fec284993816ab5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfa38318eab49ecb69a57dc81508726" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9fa5cb6c5034f9290343a342e360622" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 42417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa6eccb1153546f08fec284993816ab5" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 45937 } health_report { overall_health: HEALTHY } } }
I20260812 08:02:04.693138  5690 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 0.0 0.0
W20260812 08:02:04.716071 13316 tablet.cc:2406] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:02:04.796141  5690 cluster_itest_util.cc:258] Not converged past 1 yet: 1.1 0.0 0.0
W20260812 08:02:04.834873 13751 tablet.cc:2406] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:02:04.892999 13542 tablet.cc:2406] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:02:04.999004  5690 cluster_itest_util.cc:258] Not converged past 1 yet: 1.1 0.0 0.0
I20260812 08:02:05.153182 13792 consensus_queue.cc:1048] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dfa38318eab49ecb69a57dc81508726" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:02:05.158934 13801 consensus_queue.cc:1048] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9fa5cb6c5034f9290343a342e360622" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 42417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:02:05.302011  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 13319
I20260812 08:02:05.322324  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 13544
I20260812 08:02:05.344650 13243 consensus_queue.cc:237] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9dfa38318eab49ecb69a57dc81508726" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33535 } } peers { permanent_uuid: "fa6eccb1153546f08fec284993816ab5" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 45937 } }
W20260812 08:02:05.346208 13141 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.142.131:33535: connect: Connection refused (error 111)
W20260812 08:02:05.347183 13141 consensus_peers.cc:597] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5 -> Peer 9dfa38318eab49ecb69a57dc81508726 (127.5.142.131:33535): Couldn't send request to peer 9dfa38318eab49ecb69a57dc81508726. Status: Network error: Client connection negotiation failed: client connection to 127.5.142.131:33535: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 08:02:06.355464  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.130:42417
--local_ip_for_outbound_sockets=127.5.142.130
--tserver_master_addrs=127.5.142.190:41149
--webserver_port=40739
--webserver_interface=127.5.142.130
--builtin_ntp_servers=127.5.142.148:34913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false with env {}
W20260812 08:02:06.463544 13828 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 08:02:06.463771 13828 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:06.463801 13828 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:06.463857 13828 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:06.467150 13828 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:06.467269 13828 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.130
I20260812 08:02:06.471027 13828 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:34913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.130:42417
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.5.142.130
--webserver_port=40739
--tserver_master_addrs=127.5.142.190:41149
--never_fsync=true
--heap_profile_path=/tmp/kudu.13828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.130

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:06.472134 13828 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:06.473281 13828 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:06.479568 13839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:06.479564 13835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:06.479564 13836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:06.479907 13828 server_base.cc:1061] running on GCE node
I20260812 08:02:06.480325 13828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:06.480831 13828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:06.482000 13828 hybrid_clock.cc:648] HybridClock initialized: now 1786521726481985 us; error 25 us; skew 500 ppm
I20260812 08:02:06.483752 13828 webserver.cc:533] Webserver started at http://127.5.142.130:40739/ using document root <none> and password file <none>
I20260812 08:02:06.484295 13828 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:06.484369 13828 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:06.487622 13828 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:06.489442 13848 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:06.490350 13828 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:06.490449 13828 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "d9fa5cb6c5034f9290343a342e360622"
format_stamp: "Formatted at 2026-08-12 08:02:04 on dist-test-slave-ncp9"
I20260812 08:02:06.490834 13828 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:06.505502 13828 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:06.506203 13828 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:06.506390 13828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:06.506997 13828 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:06.508270 13856 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:02:06.511747 13828 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:02:06.511832 13828 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 08:02:06.511901 13828 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:02:06.513535 13828 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:02:06.513594 13828 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:02:06.513886 13856 tablet_bootstrap.cc:492] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622: Bootstrap starting.
I20260812 08:02:06.525523 13828 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.130:42417
I20260812 08:02:06.525574 14048 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.130:42417 every 8 connection(s)
I20260812 08:02:06.526610 13828 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:02:06.527863 13856 log.cc:826] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:06.530504 13856 tablet_bootstrap.cc:492] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:02:06.530838 13856 tablet_bootstrap.cc:492] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622: Bootstrap complete.
I20260812 08:02:06.531801 13856 ts_tablet_manager.cc:1403] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.005s
I20260812 08:02:06.533162  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 13828
I20260812 08:02:06.533573  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.131:33535
--local_ip_for_outbound_sockets=127.5.142.131
--tserver_master_addrs=127.5.142.190:41149
--webserver_port=46297
--webserver_interface=127.5.142.131
--builtin_ntp_servers=127.5.142.148:34913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false with env {}
I20260812 08:02:06.534440 13856 raft_consensus.cc:359] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfa38318eab49ecb69a57dc81508726" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33535 } } peers { permanent_uuid: "d9fa5cb6c5034f9290343a342e360622" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 42417 } } peers { permanent_uuid: "fa6eccb1153546f08fec284993816ab5" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 45937 } }
I20260812 08:02:06.534783 13856 raft_consensus.cc:740] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9fa5cb6c5034f9290343a342e360622, State: Initialized, Role: FOLLOWER
I20260812 08:02:06.535107 13856 consensus_queue.cc:260] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfa38318eab49ecb69a57dc81508726" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33535 } } peers { permanent_uuid: "d9fa5cb6c5034f9290343a342e360622" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 42417 } } peers { permanent_uuid: "fa6eccb1153546f08fec284993816ab5" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 45937 } }
I20260812 08:02:06.535979 13856 ts_tablet_manager.cc:1434] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622: Time spent starting tablet: real 0.004s	user 0.003s	sys 0.004s
I20260812 08:02:06.538033 14051 heartbeater.cc:344] Connected to a master server at 127.5.142.190:41149
I20260812 08:02:06.538268 14051 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:06.538790 14051 heartbeater.cc:507] Master 127.5.142.190:41149 requested a full tablet report, sending...
I20260812 08:02:06.540632 13000 ts_manager.cc:194] Re-registered known tserver with Master: d9fa5cb6c5034f9290343a342e360622 (127.5.142.130:42417)
I20260812 08:02:06.541476 13000 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.130:39899
I20260812 08:02:06.543181 14051 heartbeater.cc:499] Master 127.5.142.190:41149 was elected leader, sending a full tablet report...
W20260812 08:02:06.643116 14071 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 08:02:06.643342 14071 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:06.643374 14071 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:06.643440 14071 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:06.646782 14071 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:06.646903 14071 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.131
I20260812 08:02:06.650964 14071 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:34913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.131:33535
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.5.142.131
--webserver_port=46297
--tserver_master_addrs=127.5.142.190:41149
--never_fsync=true
--heap_profile_path=/tmp/kudu.14071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.131

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:06.652091 14071 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:06.653335 14071 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:06.659922 14080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:06.660007 14085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:06.659920 14081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:06.660223 14071 server_base.cc:1061] running on GCE node
I20260812 08:02:06.660537 14071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:06.661198 14071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:06.662429 14071 hybrid_clock.cc:648] HybridClock initialized: now 1786521726662395 us; error 32 us; skew 500 ppm
I20260812 08:02:06.664331 14071 webserver.cc:533] Webserver started at http://127.5.142.131:46297/ using document root <none> and password file <none>
I20260812 08:02:06.664980 14071 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:06.665062 14071 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:06.668601 14071 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:02:06.670679 14093 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:06.671700 14071 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:02:06.671800 14071 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "9dfa38318eab49ecb69a57dc81508726"
format_stamp: "Formatted at 2026-08-12 08:02:04 on dist-test-slave-ncp9"
I20260812 08:02:06.672183 14071 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:06.695590 14071 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:06.696254 14071 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:06.696408 14071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:06.696978 14071 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:06.698191 14101 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:02:06.701617 14071 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:02:06.701676 14071 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 08:02:06.701728 14071 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:02:06.703195 14071 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:02:06.703238 14071 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 08:02:06.703588 14101 tablet_bootstrap.cc:492] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726: Bootstrap starting.
I20260812 08:02:06.713032 14071 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.131:33535
I20260812 08:02:06.713085 14304 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.131:33535 every 8 connection(s)
I20260812 08:02:06.714017 14071 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:02:06.718477 14101 log.cc:826] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:06.721025 14101 tablet_bootstrap.cc:492] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:02:06.721376 14101 tablet_bootstrap.cc:492] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726: Bootstrap complete.
I20260812 08:02:06.722084  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 14071
I20260812 08:02:06.722642 14101 ts_tablet_manager.cc:1403] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.008s
I20260812 08:02:06.722770 14306 heartbeater.cc:344] Connected to a master server at 127.5.142.190:41149
I20260812 08:02:06.723027 14306 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:06.723635 14306 heartbeater.cc:507] Master 127.5.142.190:41149 requested a full tablet report, sending...
I20260812 08:02:06.725577 13000 ts_manager.cc:194] Re-registered known tserver with Master: 9dfa38318eab49ecb69a57dc81508726 (127.5.142.131:33535)
I20260812 08:02:06.725700 14101 raft_consensus.cc:359] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfa38318eab49ecb69a57dc81508726" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33535 } } peers { permanent_uuid: "d9fa5cb6c5034f9290343a342e360622" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 42417 } } peers { permanent_uuid: "fa6eccb1153546f08fec284993816ab5" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 45937 } }
I20260812 08:02:06.726078 14101 raft_consensus.cc:740] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dfa38318eab49ecb69a57dc81508726, State: Initialized, Role: FOLLOWER
I20260812 08:02:06.726390 14101 consensus_queue.cc:260] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfa38318eab49ecb69a57dc81508726" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33535 } } peers { permanent_uuid: "d9fa5cb6c5034f9290343a342e360622" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 42417 } } peers { permanent_uuid: "fa6eccb1153546f08fec284993816ab5" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 45937 } }
I20260812 08:02:06.726565 13000 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.131:50533
I20260812 08:02:06.727278 14101 ts_tablet_manager.cc:1434] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260812 08:02:06.728654 14306 heartbeater.cc:499] Master 127.5.142.190:41149 was elected leader, sending a full tablet report...
I20260812 08:02:06.735827 13971 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "625690021b4c4ccf96b681a7bea2e8a8"
dest_uuid: "d9fa5cb6c5034f9290343a342e360622"
 from {username='slave'} at 127.0.0.1:46678
I20260812 08:02:06.736025 13971 raft_consensus.cc:493] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 08:02:06.736114 13971 raft_consensus.cc:3060] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622 [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:02:06.737876 13971 raft_consensus.cc:515] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfa38318eab49ecb69a57dc81508726" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33535 } } peers { permanent_uuid: "d9fa5cb6c5034f9290343a342e360622" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 42417 } } peers { permanent_uuid: "fa6eccb1153546f08fec284993816ab5" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 45937 } }
I20260812 08:02:06.738862 13971 leader_election.cc:290] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622 [CANDIDATE]: Term 2 election: Requested vote from peers 9dfa38318eab49ecb69a57dc81508726 (127.5.142.131:33535), fa6eccb1153546f08fec284993816ab5 (127.5.142.129:45937)
I20260812 08:02:06.742846 14230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625690021b4c4ccf96b681a7bea2e8a8" candidate_uuid: "d9fa5cb6c5034f9290343a342e360622" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9dfa38318eab49ecb69a57dc81508726"
I20260812 08:02:06.743090 14230 raft_consensus.cc:3060] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726 [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:02:06.744781 14230 raft_consensus.cc:2468] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9fa5cb6c5034f9290343a342e360622 in term 2.
I20260812 08:02:06.745251 13852 leader_election.cc:304] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9dfa38318eab49ecb69a57dc81508726, d9fa5cb6c5034f9290343a342e360622; no voters: 
I20260812 08:02:06.745536 14070 raft_consensus.cc:2804] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 08:02:06.745797 14070 raft_consensus.cc:697] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622 [term 2 LEADER]: Becoming Leader. State: Replica: d9fa5cb6c5034f9290343a342e360622, State: Running, Role: LEADER
I20260812 08:02:06.746098 14070 consensus_queue.cc:237] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfa38318eab49ecb69a57dc81508726" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33535 } } peers { permanent_uuid: "d9fa5cb6c5034f9290343a342e360622" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 42417 } } peers { permanent_uuid: "fa6eccb1153546f08fec284993816ab5" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 45937 } }
I20260812 08:02:06.749188 13000 catalog_manager.cc:5719] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622 reported cstate change: term changed from 1 to 2, leader changed from fa6eccb1153546f08fec284993816ab5 (127.5.142.129) to d9fa5cb6c5034f9290343a342e360622 (127.5.142.130). New cstate: current_term: 2 leader_uuid: "d9fa5cb6c5034f9290343a342e360622" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dfa38318eab49ecb69a57dc81508726" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9fa5cb6c5034f9290343a342e360622" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 42417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa6eccb1153546f08fec284993816ab5" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 45937 } health_report { overall_health: UNKNOWN } } }
I20260812 08:02:06.757745 13243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625690021b4c4ccf96b681a7bea2e8a8" candidate_uuid: "d9fa5cb6c5034f9290343a342e360622" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa6eccb1153546f08fec284993816ab5"
I20260812 08:02:06.757974 13243 raft_consensus.cc:1813] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5 [term 1 LEADER]: Handling vote request from an unknown peer d9fa5cb6c5034f9290343a342e360622
I20260812 08:02:06.758055 13243 raft_consensus.cc:3055] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 08:02:06.758096 13243 raft_consensus.cc:740] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fa6eccb1153546f08fec284993816ab5, State: Running, Role: LEADER
I20260812 08:02:06.758256 13243 consensus_queue.cc:260] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "9dfa38318eab49ecb69a57dc81508726" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33535 } } peers { permanent_uuid: "fa6eccb1153546f08fec284993816ab5" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 45937 } }
I20260812 08:02:06.758314 14230 raft_consensus.cc:1240] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726 [term 2 FOLLOWER]: Rejecting Update request from peer fa6eccb1153546f08fec284993816ab5 for earlier term 1. Current term is 2. Ops: [1.2-1.2]
I20260812 08:02:06.758385 13243 raft_consensus.cc:3060] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5 [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:02:06.759572 13243 raft_consensus.cc:2410] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d9fa5cb6c5034f9290343a342e360622 for term 2 because replica has last-logged OpId of term: 1 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260812 08:02:06.768651 14230 raft_consensus.cc:1275] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726 [term 2 FOLLOWER]: Refusing update from remote peer d9fa5cb6c5034f9290343a342e360622: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260812 08:02:06.768765 13243 raft_consensus.cc:1275] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5 [term 2 FOLLOWER]: Refusing update from remote peer d9fa5cb6c5034f9290343a342e360622: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260812 08:02:06.769141 14070 consensus_queue.cc:1048] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dfa38318eab49ecb69a57dc81508726" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 33535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:02:06.769357 14323 consensus_queue.cc:1048] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa6eccb1153546f08fec284993816ab5" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 45937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:02:06.769471 14323 consensus_queue.cc:1243] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622 [LEADER]: Peer fa6eccb1153546f08fec284993816ab5 log is divergent from this leader: its last log entry 1.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260812 08:02:06.770582 13243 pending_rounds.cc:85] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5: Aborting all ops after (but not including) 1
I20260812 08:02:06.770665 13243 pending_rounds.cc:107] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 1.2
I20260812 08:02:06.770727 13243 raft_consensus.cc:2917] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5 [term 2 FOLLOWER]: Aborting config change with OpId 1.2: Aborted: Op aborted by new leader
I20260812 08:02:06.770802 13243 raft_consensus.cc:2889] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5 [term 2 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
W20260812 08:02:06.770978 13243 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:51960 (request call id 5) took 1427 ms (client timeout 1000 ms). Trace:
W20260812 08:02:06.771030 13243 rpcz_store.cc:269] 0812 08:02:05.343672 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:02:05.343765 (+    93us) service_pool.cc:224] Handling call
0812 08:02:05.345236 (+  1471us) log.cc:844] Serialized 379 byte log entry
0812 08:02:06.770963 (+1425727us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":54,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":144}
I20260812 08:02:06.775758 14339 mvcc.cc:204] Tried to move back new op lower bound from 7317592992838463488 to 7317592992755130368. Current Snapshot: MvccSnapshot[applied={T|T < 7317592992838463488}]
I20260812 08:02:06.777869 14340 mvcc.cc:204] Tried to move back new op lower bound from 7317592992838463488 to 7317592992755130368. Current Snapshot: MvccSnapshot[applied={T|T < 7317592992838463488}]
I20260812 08:02:06.778916 14341 mvcc.cc:204] Tried to move back new op lower bound from 7317592992838463488 to 7317592992755130368. Current Snapshot: MvccSnapshot[applied={T|T < 7317592992838463488}]
I20260812 08:02:06.824988 14189 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 08:02:06.825021 13207 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 08:02:06.825685 13937 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
Master Summary
               UUID               |       Address       | Status
----------------------------------+---------------------+---------
 d8011dd427634b0ab68b77d80ab46f2f | 127.5.142.190:41149 | HEALTHY

Unusual flags for Master:
                         Flag                         |                                                                                          Value                                                                                           |      Tags       |         Master
------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
 catalog_manager_wait_for_new_tablets_to_elect_leader | false                                                                                                                                                                                    | hidden          | all 1 server(s) checked
 ipki_ca_key_size                                     | 768                                                                                                                                                                                      | experimental    | all 1 server(s) checked
 ipki_server_key_size                                 | 768                                                                                                                                                                                      | experimental    | all 1 server(s) checked
 master_add_server_when_underreplicated               | false                                                                                                                                                                                    | hidden          | all 1 server(s) checked
 never_fsync                                          | true                                                                                                                                                                                     | unsafe,advanced | all 1 server(s) checked
 openssl_security_level_override                      | 0                                                                                                                                                                                        | unsafe,hidden   | all 1 server(s) checked
 rpc_reuseport                                        | true                                                                                                                                                                                     | experimental    | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports                     | true                                                                                                                                                                                     | unsafe          | all 1 server(s) checked
 server_dump_info_format                              | pb                                                                                                                                                                                       | hidden          | all 1 server(s) checked
 server_dump_info_path                                | /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden          | all 1 server(s) checked
 tsk_num_rsa_bits                                     | 512                                                                                                                                                                                      | experimental    | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |        Value        |         Master
---------------------+---------------------+-------------------------
 builtin_ntp_servers | 127.5.142.148:34913 | all 1 server(s) checked
 time_source         | builtin             | all 1 server(s) checked

Tablet Server Summary
               UUID               |       Address       | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
 9dfa38318eab49ecb69a57dc81508726 | 127.5.142.131:33535 | HEALTHY | <none>   |       0        |       0
 d9fa5cb6c5034f9290343a342e360622 | 127.5.142.130:42417 | HEALTHY | <none>   |       1        |       0
 fa6eccb1153546f08fec284993816ab5 | 127.5.142.129:45937 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
               Flag               |                                                                                        Value                                                                                         |      Tags       |      Tablet Server
----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
 enable_leader_failure_detection  | false                                                                                                                                                                                | unsafe          | all 3 server(s) checked
 ipki_server_key_size             | 768                                                                                                                                                                                  | experimental    | all 3 server(s) checked
 local_ip_for_outbound_sockets    | 127.5.142.129                                                                                                                                                                        | experimental    | 127.5.142.129:45937
 local_ip_for_outbound_sockets    | 127.5.142.130                                                                                                                                                                        | experimental    | 127.5.142.130:42417
 local_ip_for_outbound_sockets    | 127.5.142.131                                                                                                                                                                        | experimental    | 127.5.142.131:33535
 never_fsync                      | true                                                                                                                                                                                 | unsafe,advanced | all 3 server(s) checked
 openssl_security_level_override  | 0                                                                                                                                                                                    | unsafe,hidden   | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports | true                                                                                                                                                                                 | unsafe          | all 3 server(s) checked
 server_dump_info_format          | pb                                                                                                                                                                                   | hidden          | all 3 server(s) checked
 server_dump_info_path            | /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden          | 127.5.142.129:45937
 server_dump_info_path            | /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden          | 127.5.142.130:42417
 server_dump_info_path            | /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden          | 127.5.142.131:33535

Flags of checked categories for Tablet Server:
        Flag         |        Value        |      Tablet Server
---------------------+---------------------+-------------------------
 builtin_ntp_servers | 127.5.142.148:34913 | all 3 server(s) checked
 time_source         | builtin             | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
   Name    | RF | Status  | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-----------+----+---------+---------------+---------+------------+------------------+-------------
 TestTable | 3  | HEALTHY | 1             | 1       | 0          | 0                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

OK
I20260812 08:02:06.848431  5690 log_verifier.cc:126] Checking tablet 625690021b4c4ccf96b681a7bea2e8a8
I20260812 08:02:06.871698  5690 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 625690021b4c4ccf96b681a7bea2e8a8
I20260812 08:02:06.891887 13971 consensus_queue.cc:237] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9fa5cb6c5034f9290343a342e360622" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 42417 } } peers { permanent_uuid: "fa6eccb1153546f08fec284993816ab5" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 45937 } }
I20260812 08:02:06.893193 13243 raft_consensus.cc:1275] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5 [term 2 FOLLOWER]: Refusing update from remote peer d9fa5cb6c5034f9290343a342e360622: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260812 08:02:06.893585 14323 consensus_queue.cc:1048] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa6eccb1153546f08fec284993816ab5" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 45937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 08:02:06.895247 14070 raft_consensus.cc:2955] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index -1 to 4, VOTER 9dfa38318eab49ecb69a57dc81508726 (127.5.142.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9fa5cb6c5034f9290343a342e360622" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 42417 } } peers { permanent_uuid: "fa6eccb1153546f08fec284993816ab5" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 45937 } } }
I20260812 08:02:06.895779 13243 raft_consensus.cc:2955] T 625690021b4c4ccf96b681a7bea2e8a8 P fa6eccb1153546f08fec284993816ab5 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index -1 to 4, VOTER 9dfa38318eab49ecb69a57dc81508726 (127.5.142.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9fa5cb6c5034f9290343a342e360622" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 42417 } } peers { permanent_uuid: "fa6eccb1153546f08fec284993816ab5" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 45937 } } }
I20260812 08:02:06.898068 12995 catalog_manager.cc:5719] T 625690021b4c4ccf96b681a7bea2e8a8 P d9fa5cb6c5034f9290343a342e360622 reported cstate change: config changed from index -1 to 4, VOTER 9dfa38318eab49ecb69a57dc81508726 (127.5.142.131) evicted. New cstate: current_term: 2 leader_uuid: "d9fa5cb6c5034f9290343a342e360622" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9fa5cb6c5034f9290343a342e360622" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 42417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa6eccb1153546f08fec284993816ab5" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 45937 } health_report { overall_health: HEALTHY } } }
I20260812 08:02:06.905728 14189 tablet_service.cc:1558] Processing DeleteTablet for tablet 625690021b4c4ccf96b681a7bea2e8a8 with delete_type TABLET_DATA_TOMBSTONED (TS 9dfa38318eab49ecb69a57dc81508726 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49476
I20260812 08:02:06.906847 14406 tablet_replica.cc:333] stopping tablet replica
I20260812 08:02:06.907088  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 13103
I20260812 08:02:06.907166 14406 raft_consensus.cc:2243] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260812 08:02:06.907431 14406 raft_consensus.cc:2272] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 08:02:06.912537 14406 ts_tablet_manager.cc:1916] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 08:02:06.915099 14406 ts_tablet_manager.cc:1929] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260812 08:02:06.915189 14406 log.cc:1199] T 625690021b4c4ccf96b681a7bea2e8a8 P 9dfa38318eab49ecb69a57dc81508726: Deleting WAL directory at /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal/wals/625690021b4c4ccf96b681a7bea2e8a8
I20260812 08:02:06.915843 12979 catalog_manager.cc:5050] TS 9dfa38318eab49ecb69a57dc81508726 (127.5.142.131:33535): tablet 625690021b4c4ccf96b681a7bea2e8a8 (table TestTable [id=2b7a3bcd281341529b1920b5091f6ee7]) successfully deleted
I20260812 08:02:06.923597  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 13828
I20260812 08:02:06.942770  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 14071
I20260812 08:02:06.962387  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 12947
2026-08-12T08:02:06Z chronyd exiting
[       OK ] RaftConsensusITest.TestReplaceChangeConfigOperation (3206 ms)
[ RUN      ] RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers
I20260812 08:02:06.989135  5690 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:02:06.989192  5690 ts_itest-base.cc:116] --------------
I20260812 08:02:06.989203  5690 ts_itest-base.cc:117] 3 tablet servers
I20260812 08:02:06.989212  5690 ts_itest-base.cc:118] 3 replicas per TS
I20260812 08:02:06.989221  5690 ts_itest-base.cc:119] --------------
2026-08-12T08:02:06Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:02:06Z Disabled control of system clock
I20260812 08:02:06.999126  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.142.190:43613
--webserver_interface=127.5.142.190
--webserver_port=0
--builtin_ntp_servers=127.5.142.148:35291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.142.190:43613 with env {}
W20260812 08:02:07.106714 14419 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:07.106966 14419 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:07.107020 14419 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:07.110565 14419 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:02:07.110642 14419 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:07.110669 14419 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:02:07.110694 14419 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:02:07.115567 14419 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:35291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.142.190:43613
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.142.190:43613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.5.142.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.14419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:07.117065 14419 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:07.118391 14419 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:07.124483 14428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:07.124475 14431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:07.124469 14433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:07.124781 14419 server_base.cc:1061] running on GCE node
I20260812 08:02:07.125349 14419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:07.126377 14419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:07.127542 14419 hybrid_clock.cc:648] HybridClock initialized: now 1786521727127524 us; error 31 us; skew 500 ppm
I20260812 08:02:07.129459 14419 webserver.cc:533] Webserver started at http://127.5.142.190:42747/ using document root <none> and password file <none>
I20260812 08:02:07.130053 14419 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:07.130115 14419 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:07.130329 14419 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:07.131954 14419 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "30aa757830024af6837df4a8a20e90b7"
format_stamp: "Formatted at 2026-08-12 08:02:07 on dist-test-slave-ncp9"
I20260812 08:02:07.132444 14419 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "30aa757830024af6837df4a8a20e90b7"
format_stamp: "Formatted at 2026-08-12 08:02:07 on dist-test-slave-ncp9"
I20260812 08:02:07.135874 14419 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.006s	sys 0.000s
I20260812 08:02:07.138139 14443 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:07.139215 14419 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:02:07.139329 14419 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
uuid: "30aa757830024af6837df4a8a20e90b7"
format_stamp: "Formatted at 2026-08-12 08:02:07 on dist-test-slave-ncp9"
I20260812 08:02:07.139420 14419 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:07.158316 14419 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:07.159066 14419 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:07.159263 14419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:07.167341 14524 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.190:43613 every 8 connection(s)
I20260812 08:02:07.167357 14419 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.190:43613
I20260812 08:02:07.168488 14419 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:02:07.171463 14526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:07.175079  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 14419
I20260812 08:02:07.175200  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:02:07.176730 14526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30aa757830024af6837df4a8a20e90b7: Bootstrap starting.
I20260812 08:02:07.179349 14526 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30aa757830024af6837df4a8a20e90b7: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:07.180241 14526 log.cc:826] T 00000000000000000000000000000000 P 30aa757830024af6837df4a8a20e90b7: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:07.182251 14526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30aa757830024af6837df4a8a20e90b7: No bootstrap required, opened a new log
I20260812 08:02:07.185201 14526 raft_consensus.cc:359] T 00000000000000000000000000000000 P 30aa757830024af6837df4a8a20e90b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30aa757830024af6837df4a8a20e90b7" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 43613 } }
I20260812 08:02:07.185402 14526 raft_consensus.cc:385] T 00000000000000000000000000000000 P 30aa757830024af6837df4a8a20e90b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:07.185446 14526 raft_consensus.cc:740] T 00000000000000000000000000000000 P 30aa757830024af6837df4a8a20e90b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30aa757830024af6837df4a8a20e90b7, State: Initialized, Role: FOLLOWER
I20260812 08:02:07.185794 14526 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30aa757830024af6837df4a8a20e90b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30aa757830024af6837df4a8a20e90b7" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 43613 } }
I20260812 08:02:07.185895 14526 raft_consensus.cc:399] T 00000000000000000000000000000000 P 30aa757830024af6837df4a8a20e90b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:07.185936 14526 raft_consensus.cc:493] T 00000000000000000000000000000000 P 30aa757830024af6837df4a8a20e90b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:07.186017 14526 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 30aa757830024af6837df4a8a20e90b7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:07.186743 14526 raft_consensus.cc:515] T 00000000000000000000000000000000 P 30aa757830024af6837df4a8a20e90b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30aa757830024af6837df4a8a20e90b7" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 43613 } }
I20260812 08:02:07.187032 14526 leader_election.cc:304] T 00000000000000000000000000000000 P 30aa757830024af6837df4a8a20e90b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30aa757830024af6837df4a8a20e90b7; no voters: 
I20260812 08:02:07.187273 14526 leader_election.cc:290] T 00000000000000000000000000000000 P 30aa757830024af6837df4a8a20e90b7 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:02:07.187378 14533 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 30aa757830024af6837df4a8a20e90b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:02:07.187570 14533 raft_consensus.cc:697] T 00000000000000000000000000000000 P 30aa757830024af6837df4a8a20e90b7 [term 1 LEADER]: Becoming Leader. State: Replica: 30aa757830024af6837df4a8a20e90b7, State: Running, Role: LEADER
I20260812 08:02:07.187861 14533 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30aa757830024af6837df4a8a20e90b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30aa757830024af6837df4a8a20e90b7" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 43613 } }
I20260812 08:02:07.188323 14526 sys_catalog.cc:565] T 00000000000000000000000000000000 P 30aa757830024af6837df4a8a20e90b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:02:07.189422 14537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30aa757830024af6837df4a8a20e90b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30aa757830024af6837df4a8a20e90b7. Latest consensus state: current_term: 1 leader_uuid: "30aa757830024af6837df4a8a20e90b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30aa757830024af6837df4a8a20e90b7" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 43613 } } }
I20260812 08:02:07.189572 14537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30aa757830024af6837df4a8a20e90b7 [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:07.189836 14535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30aa757830024af6837df4a8a20e90b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30aa757830024af6837df4a8a20e90b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30aa757830024af6837df4a8a20e90b7" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 43613 } } }
I20260812 08:02:07.189924 14535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30aa757830024af6837df4a8a20e90b7 [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:07.190732 14551 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:02:07.193504 14551 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:02:07.199527 14551 catalog_manager.cc:1383] Generated new cluster ID: 5829c70b6be848a7816c6496e9ee2854
I20260812 08:02:07.199616 14551 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:02:07.209426 14551 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:02:07.210536 14551 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:02:07.226729 14551 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 30aa757830024af6837df4a8a20e90b7: Generated new TSK 0
I20260812 08:02:07.227505 14551 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:02:07.237263  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.129:0
--local_ip_for_outbound_sockets=127.5.142.129
--webserver_interface=127.5.142.129
--webserver_port=0
--tserver_master_addrs=127.5.142.190:43613
--builtin_ntp_servers=127.5.142.148:35291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tablet_transaction_memory_limit_mb=2
--rpc_max_message_size=2097152 with env {}
W20260812 08:02:07.344396 14567 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:07.344596 14567 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:07.344687 14567 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:07.348031 14567 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:07.348165 14567 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.129
I20260812 08:02:07.352006 14567 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:35291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
--rpc_max_message_size=2097152
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.5.142.129
--webserver_port=0
--tablet_transaction_memory_limit_mb=2
--tserver_master_addrs=127.5.142.190:43613
--never_fsync=true
--heap_profile_path=/tmp/kudu.14567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.129

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:07.353132 14567 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:07.354311 14567 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:07.360333 14580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:07.360412 14577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:07.360508 14576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:07.360719 14567 server_base.cc:1061] running on GCE node
I20260812 08:02:07.361120 14567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:07.361620 14567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:07.362771 14567 hybrid_clock.cc:648] HybridClock initialized: now 1786521727362754 us; error 29 us; skew 500 ppm
I20260812 08:02:07.364455 14567 webserver.cc:533] Webserver started at http://127.5.142.129:36107/ using document root <none> and password file <none>
I20260812 08:02:07.365034 14567 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:07.365097 14567 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:07.365294 14567 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:07.366789 14567 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "7110b16d72fa4160aee65ec4a94b1817"
format_stamp: "Formatted at 2026-08-12 08:02:07 on dist-test-slave-ncp9"
I20260812 08:02:07.367242 14567 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "7110b16d72fa4160aee65ec4a94b1817"
format_stamp: "Formatted at 2026-08-12 08:02:07 on dist-test-slave-ncp9"
I20260812 08:02:07.370440 14567 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:02:07.372550 14590 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:07.373535 14567 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 08:02:07.373641 14567 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "7110b16d72fa4160aee65ec4a94b1817"
format_stamp: "Formatted at 2026-08-12 08:02:07 on dist-test-slave-ncp9"
I20260812 08:02:07.373728 14567 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:07.384176 14567 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:07.384810 14567 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:07.385010 14567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:07.385574 14567 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:07.386592 14567 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:07.386646 14567 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:07.386700 14567 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:07.386729 14567 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:07.395776 14567 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.129:35289
I20260812 08:02:07.395807 14760 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.129:35289 every 8 connection(s)
I20260812 08:02:07.396709 14567 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:02:07.404836 14768 heartbeater.cc:344] Connected to a master server at 127.5.142.190:43613
I20260812 08:02:07.405120  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 14567
I20260812 08:02:07.405149 14768 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:07.405246  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:02:07.405808 14768 heartbeater.cc:507] Master 127.5.142.190:43613 requested a full tablet report, sending...
I20260812 08:02:07.407397 14468 ts_manager.cc:194] Registered new tserver with Master: 7110b16d72fa4160aee65ec4a94b1817 (127.5.142.129:35289)
I20260812 08:02:07.407711  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.130:0
--local_ip_for_outbound_sockets=127.5.142.130
--webserver_interface=127.5.142.130
--webserver_port=0
--tserver_master_addrs=127.5.142.190:43613
--builtin_ntp_servers=127.5.142.148:35291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tablet_transaction_memory_limit_mb=2
--rpc_max_message_size=2097152 with env {}
I20260812 08:02:07.408946 14468 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.129:49351
W20260812 08:02:07.518090 14774 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:07.518324 14774 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:07.518392 14774 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:07.521798 14774 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:07.521940 14774 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.130
I20260812 08:02:07.525931 14774 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:35291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
--rpc_max_message_size=2097152
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.5.142.130
--webserver_port=0
--tablet_transaction_memory_limit_mb=2
--tserver_master_addrs=127.5.142.190:43613
--never_fsync=true
--heap_profile_path=/tmp/kudu.14774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.130

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:07.527087 14774 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:07.528221 14774 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:07.535163 14781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:07.535152 14782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:07.535154 14784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:07.535404 14774 server_base.cc:1061] running on GCE node
I20260812 08:02:07.535992 14774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:07.536571 14774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:07.537727 14774 hybrid_clock.cc:648] HybridClock initialized: now 1786521727537707 us; error 30 us; skew 500 ppm
I20260812 08:02:07.539561 14774 webserver.cc:533] Webserver started at http://127.5.142.130:46339/ using document root <none> and password file <none>
I20260812 08:02:07.540158 14774 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:07.540213 14774 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:07.540413 14774 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:07.542052 14774 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "3807fe8e93da4375a1db94d7214238c6"
format_stamp: "Formatted at 2026-08-12 08:02:07 on dist-test-slave-ncp9"
I20260812 08:02:07.542549 14774 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "3807fe8e93da4375a1db94d7214238c6"
format_stamp: "Formatted at 2026-08-12 08:02:07 on dist-test-slave-ncp9"
I20260812 08:02:07.545957 14774 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:02:07.548117 14791 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:07.549151 14774 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:02:07.549255 14774 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "3807fe8e93da4375a1db94d7214238c6"
format_stamp: "Formatted at 2026-08-12 08:02:07 on dist-test-slave-ncp9"
I20260812 08:02:07.549338 14774 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:07.566202 14774 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:07.566870 14774 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:07.567052 14774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:07.567637 14774 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:07.568687 14774 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:07.568738 14774 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:07.568792 14774 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:07.568816 14774 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:07.577646 14774 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.130:39203
I20260812 08:02:07.577694 14978 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.130:39203 every 8 connection(s)
I20260812 08:02:07.578730 14774 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:02:07.585489  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 14774
I20260812 08:02:07.585618  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:02:07.586952 14979 heartbeater.cc:344] Connected to a master server at 127.5.142.190:43613
I20260812 08:02:07.587199 14979 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:07.587709 14979 heartbeater.cc:507] Master 127.5.142.190:43613 requested a full tablet report, sending...
I20260812 08:02:07.587939  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.131:0
--local_ip_for_outbound_sockets=127.5.142.131
--webserver_interface=127.5.142.131
--webserver_port=0
--tserver_master_addrs=127.5.142.190:43613
--builtin_ntp_servers=127.5.142.148:35291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tablet_transaction_memory_limit_mb=2
--rpc_max_message_size=2097152 with env {}
I20260812 08:02:07.588769 14468 ts_manager.cc:194] Registered new tserver with Master: 3807fe8e93da4375a1db94d7214238c6 (127.5.142.130:39203)
I20260812 08:02:07.589433 14468 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.130:52565
W20260812 08:02:07.697126 14984 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:07.697315 14984 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:07.697358 14984 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:07.700520 14984 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:07.700618 14984 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.131
I20260812 08:02:07.704283 14984 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:35291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
--rpc_max_message_size=2097152
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.5.142.131
--webserver_port=0
--tablet_transaction_memory_limit_mb=2
--tserver_master_addrs=127.5.142.190:43613
--never_fsync=true
--heap_profile_path=/tmp/kudu.14984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.131

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:07.705351 14984 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:07.706446 14984 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:07.712788 14997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:07.712824 14994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:07.712824 14991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:07.713001 14984 server_base.cc:1061] running on GCE node
I20260812 08:02:07.713424 14984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:07.713948 14984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:07.715101 14984 hybrid_clock.cc:648] HybridClock initialized: now 1786521727715086 us; error 27 us; skew 500 ppm
I20260812 08:02:07.717005 14984 webserver.cc:533] Webserver started at http://127.5.142.131:42669/ using document root <none> and password file <none>
I20260812 08:02:07.717600 14984 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:07.717665 14984 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:07.717860 14984 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:07.720659 14984 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "8ab1ff81151e45dca20172a703850a53"
format_stamp: "Formatted at 2026-08-12 08:02:07 on dist-test-slave-ncp9"
I20260812 08:02:07.721378 14984 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "8ab1ff81151e45dca20172a703850a53"
format_stamp: "Formatted at 2026-08-12 08:02:07 on dist-test-slave-ncp9"
I20260812 08:02:07.725080 14984 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:02:07.727361 15005 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:07.728379 14984 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:07.728490 14984 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "8ab1ff81151e45dca20172a703850a53"
format_stamp: "Formatted at 2026-08-12 08:02:07 on dist-test-slave-ncp9"
I20260812 08:02:07.728577 14984 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:07.751742 14984 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:07.752447 14984 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:07.752638 14984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:07.753305 14984 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:07.754411 14984 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:07.754463 14984 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:07.754518 14984 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:07.754541 14984 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:07.763222 14984 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.131:38837
I20260812 08:02:07.763264 15197 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.131:38837 every 8 connection(s)
I20260812 08:02:07.764194 14984 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:02:07.765282  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 14984
I20260812 08:02:07.765395  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:02:07.771908 15198 heartbeater.cc:344] Connected to a master server at 127.5.142.190:43613
I20260812 08:02:07.772150 15198 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:07.772642 15198 heartbeater.cc:507] Master 127.5.142.190:43613 requested a full tablet report, sending...
I20260812 08:02:07.773675 14468 ts_manager.cc:194] Registered new tserver with Master: 8ab1ff81151e45dca20172a703850a53 (127.5.142.131:38837)
I20260812 08:02:07.774190 14468 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.131:38921
I20260812 08:02:07.779142  5690 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:02:07.787138 14468 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57278:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
W20260812 08:02:07.789569 14468 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 08:02:07.806104 14864 tablet_service.cc:1511] Processing CreateTablet for tablet 3264510114344d3789ddff9f3b40f221 (DEFAULT_TABLE table=TestTable [id=55e50dd77ddb4e9e9e7c56d3c3bf3d0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:07.806172 14661 tablet_service.cc:1511] Processing CreateTablet for tablet 3264510114344d3789ddff9f3b40f221 (DEFAULT_TABLE table=TestTable [id=55e50dd77ddb4e9e9e7c56d3c3bf3d0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:07.806591 15094 tablet_service.cc:1511] Processing CreateTablet for tablet 3264510114344d3789ddff9f3b40f221 (DEFAULT_TABLE table=TestTable [id=55e50dd77ddb4e9e9e7c56d3c3bf3d0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:07.807072 14864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3264510114344d3789ddff9f3b40f221. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:07.807145 14661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3264510114344d3789ddff9f3b40f221. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:07.807418 15094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3264510114344d3789ddff9f3b40f221. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:07.812192 15234 tablet_bootstrap.cc:492] T 3264510114344d3789ddff9f3b40f221 P 3807fe8e93da4375a1db94d7214238c6: Bootstrap starting.
I20260812 08:02:07.813417 15237 tablet_bootstrap.cc:492] T 3264510114344d3789ddff9f3b40f221 P 7110b16d72fa4160aee65ec4a94b1817: Bootstrap starting.
I20260812 08:02:07.814714 15234 tablet_bootstrap.cc:654] T 3264510114344d3789ddff9f3b40f221 P 3807fe8e93da4375a1db94d7214238c6: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:07.815196 15237 tablet_bootstrap.cc:654] T 3264510114344d3789ddff9f3b40f221 P 7110b16d72fa4160aee65ec4a94b1817: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:07.815390 15234 log.cc:826] T 3264510114344d3789ddff9f3b40f221 P 3807fe8e93da4375a1db94d7214238c6: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:07.815826 15237 log.cc:826] T 3264510114344d3789ddff9f3b40f221 P 7110b16d72fa4160aee65ec4a94b1817: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:07.816525 15236 tablet_bootstrap.cc:492] T 3264510114344d3789ddff9f3b40f221 P 8ab1ff81151e45dca20172a703850a53: Bootstrap starting.
I20260812 08:02:07.817147 15234 tablet_bootstrap.cc:492] T 3264510114344d3789ddff9f3b40f221 P 3807fe8e93da4375a1db94d7214238c6: No bootstrap required, opened a new log
I20260812 08:02:07.817341 15237 tablet_bootstrap.cc:492] T 3264510114344d3789ddff9f3b40f221 P 7110b16d72fa4160aee65ec4a94b1817: No bootstrap required, opened a new log
I20260812 08:02:07.817348 15234 ts_tablet_manager.cc:1403] T 3264510114344d3789ddff9f3b40f221 P 3807fe8e93da4375a1db94d7214238c6: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:02:07.817507 15237 ts_tablet_manager.cc:1403] T 3264510114344d3789ddff9f3b40f221 P 7110b16d72fa4160aee65ec4a94b1817: Time spent bootstrapping tablet: real 0.004s	user 0.003s	sys 0.000s
I20260812 08:02:07.818608 15236 tablet_bootstrap.cc:654] T 3264510114344d3789ddff9f3b40f221 P 8ab1ff81151e45dca20172a703850a53: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:07.819396 15236 log.cc:826] T 3264510114344d3789ddff9f3b40f221 P 8ab1ff81151e45dca20172a703850a53: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:07.820084 15237 raft_consensus.cc:359] T 3264510114344d3789ddff9f3b40f221 P 7110b16d72fa4160aee65ec4a94b1817 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7110b16d72fa4160aee65ec4a94b1817" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 35289 } } peers { permanent_uuid: "8ab1ff81151e45dca20172a703850a53" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 38837 } } peers { permanent_uuid: "3807fe8e93da4375a1db94d7214238c6" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 39203 } }
I20260812 08:02:07.820282 15237 raft_consensus.cc:385] T 3264510114344d3789ddff9f3b40f221 P 7110b16d72fa4160aee65ec4a94b1817 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:07.820338 15237 raft_consensus.cc:740] T 3264510114344d3789ddff9f3b40f221 P 7110b16d72fa4160aee65ec4a94b1817 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7110b16d72fa4160aee65ec4a94b1817, State: Initialized, Role: FOLLOWER
I20260812 08:02:07.820454 15234 raft_consensus.cc:359] T 3264510114344d3789ddff9f3b40f221 P 3807fe8e93da4375a1db94d7214238c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7110b16d72fa4160aee65ec4a94b1817" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 35289 } } peers { permanent_uuid: "8ab1ff81151e45dca20172a703850a53" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 38837 } } peers { permanent_uuid: "3807fe8e93da4375a1db94d7214238c6" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 39203 } }
I20260812 08:02:07.820667 15234 raft_consensus.cc:385] T 3264510114344d3789ddff9f3b40f221 P 3807fe8e93da4375a1db94d7214238c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:07.820725 15234 raft_consensus.cc:740] T 3264510114344d3789ddff9f3b40f221 P 3807fe8e93da4375a1db94d7214238c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3807fe8e93da4375a1db94d7214238c6, State: Initialized, Role: FOLLOWER
I20260812 08:02:07.820777 15237 consensus_queue.cc:260] T 3264510114344d3789ddff9f3b40f221 P 7110b16d72fa4160aee65ec4a94b1817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7110b16d72fa4160aee65ec4a94b1817" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 35289 } } peers { permanent_uuid: "8ab1ff81151e45dca20172a703850a53" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 38837 } } peers { permanent_uuid: "3807fe8e93da4375a1db94d7214238c6" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 39203 } }
I20260812 08:02:07.821213 15236 tablet_bootstrap.cc:492] T 3264510114344d3789ddff9f3b40f221 P 8ab1ff81151e45dca20172a703850a53: No bootstrap required, opened a new log
I20260812 08:02:07.821175 15234 consensus_queue.cc:260] T 3264510114344d3789ddff9f3b40f221 P 3807fe8e93da4375a1db94d7214238c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7110b16d72fa4160aee65ec4a94b1817" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 35289 } } peers { permanent_uuid: "8ab1ff81151e45dca20172a703850a53" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 38837 } } peers { permanent_uuid: "3807fe8e93da4375a1db94d7214238c6" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 39203 } }
I20260812 08:02:07.821405 15236 ts_tablet_manager.cc:1403] T 3264510114344d3789ddff9f3b40f221 P 8ab1ff81151e45dca20172a703850a53: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:02:07.821547 14768 heartbeater.cc:499] Master 127.5.142.190:43613 was elected leader, sending a full tablet report...
I20260812 08:02:07.822026 15234 ts_tablet_manager.cc:1434] T 3264510114344d3789ddff9f3b40f221 P 3807fe8e93da4375a1db94d7214238c6: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:02:07.822388 14979 heartbeater.cc:499] Master 127.5.142.190:43613 was elected leader, sending a full tablet report...
I20260812 08:02:07.822695 15237 ts_tablet_manager.cc:1434] T 3264510114344d3789ddff9f3b40f221 P 7110b16d72fa4160aee65ec4a94b1817: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260812 08:02:07.824234 15236 raft_consensus.cc:359] T 3264510114344d3789ddff9f3b40f221 P 8ab1ff81151e45dca20172a703850a53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7110b16d72fa4160aee65ec4a94b1817" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 35289 } } peers { permanent_uuid: "8ab1ff81151e45dca20172a703850a53" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 38837 } } peers { permanent_uuid: "3807fe8e93da4375a1db94d7214238c6" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 39203 } }
I20260812 08:02:07.824397 15236 raft_consensus.cc:385] T 3264510114344d3789ddff9f3b40f221 P 8ab1ff81151e45dca20172a703850a53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:07.824462 15236 raft_consensus.cc:740] T 3264510114344d3789ddff9f3b40f221 P 8ab1ff81151e45dca20172a703850a53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ab1ff81151e45dca20172a703850a53, State: Initialized, Role: FOLLOWER
I20260812 08:02:07.824867 15236 consensus_queue.cc:260] T 3264510114344d3789ddff9f3b40f221 P 8ab1ff81151e45dca20172a703850a53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7110b16d72fa4160aee65ec4a94b1817" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 35289 } } peers { permanent_uuid: "8ab1ff81151e45dca20172a703850a53" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 38837 } } peers { permanent_uuid: "3807fe8e93da4375a1db94d7214238c6" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 39203 } }
I20260812 08:02:07.825640 15198 heartbeater.cc:499] Master 127.5.142.190:43613 was elected leader, sending a full tablet report...
I20260812 08:02:07.825713 15236 ts_tablet_manager.cc:1434] T 3264510114344d3789ddff9f3b40f221 P 8ab1ff81151e45dca20172a703850a53: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
W20260812 08:02:07.829540 14981 tablet.cc:2406] T 3264510114344d3789ddff9f3b40f221 P 3807fe8e93da4375a1db94d7214238c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:02:07.897778 14770 tablet.cc:2406] T 3264510114344d3789ddff9f3b40f221 P 7110b16d72fa4160aee65ec4a94b1817: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:02:07.991926 15243 raft_consensus.cc:493] T 3264510114344d3789ddff9f3b40f221 P 7110b16d72fa4160aee65ec4a94b1817 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:02:07.992159 15243 raft_consensus.cc:515] T 3264510114344d3789ddff9f3b40f221 P 7110b16d72fa4160aee65ec4a94b1817 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7110b16d72fa4160aee65ec4a94b1817" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 35289 } } peers { permanent_uuid: "8ab1ff81151e45dca20172a703850a53" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 38837 } } peers { permanent_uuid: "3807fe8e93da4375a1db94d7214238c6" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 39203 } }
I20260812 08:02:07.993364 15243 leader_election.cc:290] T 3264510114344d3789ddff9f3b40f221 P 7110b16d72fa4160aee65ec4a94b1817 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ab1ff81151e45dca20172a703850a53 (127.5.142.131:38837), 3807fe8e93da4375a1db94d7214238c6 (127.5.142.130:39203)
I20260812 08:02:07.997169 15125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3264510114344d3789ddff9f3b40f221" candidate_uuid: "7110b16d72fa4160aee65ec4a94b1817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ab1ff81151e45dca20172a703850a53" is_pre_election: true
I20260812 08:02:07.997514 15125 raft_consensus.cc:2468] T 3264510114344d3789ddff9f3b40f221 P 8ab1ff81151e45dca20172a703850a53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7110b16d72fa4160aee65ec4a94b1817 in term 0.
I20260812 08:02:07.997529 14894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3264510114344d3789ddff9f3b40f221" candidate_uuid: "7110b16d72fa4160aee65ec4a94b1817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3807fe8e93da4375a1db94d7214238c6" is_pre_election: true
I20260812 08:02:07.997778 14894 raft_consensus.cc:2468] T 3264510114344d3789ddff9f3b40f221 P 3807fe8e93da4375a1db94d7214238c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7110b16d72fa4160aee65ec4a94b1817 in term 0.
I20260812 08:02:07.998072 14591 leader_election.cc:304] T 3264510114344d3789ddff9f3b40f221 P 7110b16d72fa4160aee65ec4a94b1817 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7110b16d72fa4160aee65ec4a94b1817, 8ab1ff81151e45dca20172a703850a53; no voters: 
I20260812 08:02:07.998348 15243 raft_consensus.cc:2804] T 3264510114344d3789ddff9f3b40f221 P 7110b16d72fa4160aee65ec4a94b1817 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:02:07.998423 15243 raft_consensus.cc:493] T 3264510114344d3789ddff9f3b40f221 P 7110b16d72fa4160aee65ec4a94b1817 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:02:07.998472 15243 raft_consensus.cc:3060] T 3264510114344d3789ddff9f3b40f221 P 7110b16d72fa4160aee65ec4a94b1817 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:07.999369 15243 raft_consensus.cc:515] T 3264510114344d3789ddff9f3b40f221 P 7110b16d72fa4160aee65ec4a94b1817 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7110b16d72fa4160aee65ec4a94b1817" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 35289 } } peers { permanent_uuid: "8ab1ff81151e45dca20172a703850a53" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 38837 } } peers { permanent_uuid: "3807fe8e93da4375a1db94d7214238c6" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 39203 } }
I20260812 08:02:07.999724 15243 leader_election.cc:290] T 3264510114344d3789ddff9f3b40f221 P 7110b16d72fa4160aee65ec4a94b1817 [CANDIDATE]: Term 1 election: Requested vote from peers 8ab1ff81151e45dca20172a703850a53 (127.5.142.131:38837), 3807fe8e93da4375a1db94d7214238c6 (127.5.142.130:39203)
I20260812 08:02:08.000077 14894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3264510114344d3789ddff9f3b40f221" candidate_uuid: "7110b16d72fa4160aee65ec4a94b1817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3807fe8e93da4375a1db94d7214238c6"
I20260812 08:02:08.000077 15125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3264510114344d3789ddff9f3b40f221" candidate_uuid: "7110b16d72fa4160aee65ec4a94b1817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ab1ff81151e45dca20172a703850a53"
I20260812 08:02:08.000204 15125 raft_consensus.cc:3060] T 3264510114344d3789ddff9f3b40f221 P 8ab1ff81151e45dca20172a703850a53 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:08.000211 14894 raft_consensus.cc:3060] T 3264510114344d3789ddff9f3b40f221 P 3807fe8e93da4375a1db94d7214238c6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:08.001366 14894 raft_consensus.cc:2468] T 3264510114344d3789ddff9f3b40f221 P 3807fe8e93da4375a1db94d7214238c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7110b16d72fa4160aee65ec4a94b1817 in term 1.
I20260812 08:02:08.001708 15125 raft_consensus.cc:2468] T 3264510114344d3789ddff9f3b40f221 P 8ab1ff81151e45dca20172a703850a53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7110b16d72fa4160aee65ec4a94b1817 in term 1.
I20260812 08:02:08.001720 14593 leader_election.cc:304] T 3264510114344d3789ddff9f3b40f221 P 7110b16d72fa4160aee65ec4a94b1817 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3807fe8e93da4375a1db94d7214238c6, 7110b16d72fa4160aee65ec4a94b1817; no voters: 
I20260812 08:02:08.001886 15243 raft_consensus.cc:2804] T 3264510114344d3789ddff9f3b40f221 P 7110b16d72fa4160aee65ec4a94b1817 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:02:08.002139 15243 raft_consensus.cc:697] T 3264510114344d3789ddff9f3b40f221 P 7110b16d72fa4160aee65ec4a94b1817 [term 1 LEADER]: Becoming Leader. State: Replica: 7110b16d72fa4160aee65ec4a94b1817, State: Running, Role: LEADER
I20260812 08:02:08.002475 15243 consensus_queue.cc:237] T 3264510114344d3789ddff9f3b40f221 P 7110b16d72fa4160aee65ec4a94b1817 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7110b16d72fa4160aee65ec4a94b1817" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 35289 } } peers { permanent_uuid: "8ab1ff81151e45dca20172a703850a53" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 38837 } } peers { permanent_uuid: "3807fe8e93da4375a1db94d7214238c6" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 39203 } }
I20260812 08:02:08.005493 14467 catalog_manager.cc:5719] T 3264510114344d3789ddff9f3b40f221 P 7110b16d72fa4160aee65ec4a94b1817 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7110b16d72fa4160aee65ec4a94b1817 (127.5.142.129). New cstate: current_term: 1 leader_uuid: "7110b16d72fa4160aee65ec4a94b1817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7110b16d72fa4160aee65ec4a94b1817" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 35289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ab1ff81151e45dca20172a703850a53" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3807fe8e93da4375a1db94d7214238c6" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 39203 } health_report { overall_health: UNKNOWN } } }
W20260812 08:02:08.015156 15199 tablet.cc:2406] T 3264510114344d3789ddff9f3b40f221 P 8ab1ff81151e45dca20172a703850a53: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:02:08.033910  5690 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:02:08.035475  5690 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 7110b16d72fa4160aee65ec4a94b1817 to finish bootstrapping
I20260812 08:02:08.040757  5690 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 3807fe8e93da4375a1db94d7214238c6 to finish bootstrapping
I20260812 08:02:08.045328  5690 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 8ab1ff81151e45dca20172a703850a53 to finish bootstrapping
I20260812 08:02:08.054973  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 14774
I20260812 08:02:08.072252  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 14984
I20260812 08:02:08.094853  5690 test_workload.cc:405] TestWorkload: Skipping table creation because table TestTable already exists
W20260812 08:02:08.137517 14625 op_tracker.cc:165] op on tablet 3264510114344d3789ddff9f3b40f221 rejected due to memory pressure: the memory usage of this op (1502305) plus the current consumption (1502305) exceeds the op memory limit (2097152) or the limit of an ancestral memory tracker.
W20260812 08:02:08.142372 14593 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.142.130:39203: connect: Connection refused (error 111)
W20260812 08:02:08.143544 14591 consensus_peers.cc:597] T 3264510114344d3789ddff9f3b40f221 P 7110b16d72fa4160aee65ec4a94b1817 -> Peer 8ab1ff81151e45dca20172a703850a53 (127.5.142.131:38837): Couldn't send request to peer 8ab1ff81151e45dca20172a703850a53. Status: Network error: Client connection negotiation failed: client connection to 127.5.142.131:38837: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260812 08:02:08.143544 14593 consensus_peers.cc:597] T 3264510114344d3789ddff9f3b40f221 P 7110b16d72fa4160aee65ec4a94b1817 -> Peer 3807fe8e93da4375a1db94d7214238c6 (127.5.142.130:39203): Couldn't send request to peer 3807fe8e93da4375a1db94d7214238c6. Status: Network error: Client connection negotiation failed: client connection to 127.5.142.130:39203: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260812 08:02:08.262584 15263 batcher.cc:441] Timed out: Failed to write batch of 50 ops to tablet 3264510114344d3789ddff9f3b40f221 after 12 attempt(s): Failed to write to server: (no server available): Write(tablet: 3264510114344d3789ddff9f3b40f221, num_ops: 50, num_attempts: 12) passed its deadline: Remote error: Service unavailable: op on tablet 3264510114344d3789ddff9f3b40f221 rejected due to memory pressure: the memory usage of this op (1502305) plus the current consumption (1502305) exceeds the op memory limit (2097152) or the limit of an ancestral memory tracker.
W20260812 08:02:08.264407 15263 meta_cache.cc:302] tablet 3264510114344d3789ddff9f3b40f221: replica 7110b16d72fa4160aee65ec4a94b1817 (127.5.142.129:35289) has failed: Timed out: Write RPC to 127.5.142.129:35289 timed out after 0.139s (SENT)
I20260812 08:02:08.606415  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 14567
I20260812 08:02:08.623687  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 14419
2026-08-12T08:02:08Z chronyd exiting
[       OK ] RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers (1670 ms)
[ RUN      ] RaftConsensusITest.TestEvictAbandonedFollowers
I20260812 08:02:08.660233  5690 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:02:08.660287  5690 ts_itest-base.cc:116] --------------
I20260812 08:02:08.660300  5690 ts_itest-base.cc:117] 3 tablet servers
I20260812 08:02:08.660310  5690 ts_itest-base.cc:118] 3 replicas per TS
I20260812 08:02:08.660318  5690 ts_itest-base.cc:119] --------------
2026-08-12T08:02:08Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:02:08Z Disabled control of system clock
I20260812 08:02:08.672729  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.142.190:35287
--webserver_interface=127.5.142.190
--webserver_port=0
--builtin_ntp_servers=127.5.142.148:34105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.142.190:35287
--master_add_server_when_underreplicated=false
--raft_prepare_replacement_before_eviction=false with env {}
W20260812 08:02:08.780967 15295 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:08.781184 15295 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:08.781221 15295 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:08.784655 15295 flags.cc:432] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20260812 08:02:08.784722 15295 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:02:08.784741 15295 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:08.784758 15295 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:02:08.784775 15295 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:02:08.788558 15295 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:34105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
--master_add_server_when_underreplicated=false
--ipki_ca_key_size=768
--master_addresses=127.5.142.190:35287
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.142.190:35287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.5.142.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.15295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:08.789611 15295 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:08.790683 15295 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:08.795820 15303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:08.795827 15305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:08.795858 15308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:08.796123 15295 server_base.cc:1061] running on GCE node
I20260812 08:02:08.796630 15295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:08.797513 15295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:08.798666 15295 hybrid_clock.cc:648] HybridClock initialized: now 1786521728798652 us; error 30 us; skew 500 ppm
I20260812 08:02:08.800268 15295 webserver.cc:533] Webserver started at http://127.5.142.190:44679/ using document root <none> and password file <none>
I20260812 08:02:08.800806 15295 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:08.800863 15295 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:08.801065 15295 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:08.802563 15295 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "3e769cd3e5b24d8b882df46ee02d1f72"
format_stamp: "Formatted at 2026-08-12 08:02:08 on dist-test-slave-ncp9"
I20260812 08:02:08.803004 15295 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "3e769cd3e5b24d8b882df46ee02d1f72"
format_stamp: "Formatted at 2026-08-12 08:02:08 on dist-test-slave-ncp9"
I20260812 08:02:08.806143 15295 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:02:08.808120 15315 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:08.809129 15295 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:02:08.809230 15295 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
uuid: "3e769cd3e5b24d8b882df46ee02d1f72"
format_stamp: "Formatted at 2026-08-12 08:02:08 on dist-test-slave-ncp9"
I20260812 08:02:08.809309 15295 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:08.819739 15295 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:08.820356 15295 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:08.820518 15295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:08.827639 15295 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.190:35287
I20260812 08:02:08.827637 15408 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.190:35287 every 8 connection(s)
I20260812 08:02:08.828637 15295 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:02:08.829180  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 15295
I20260812 08:02:08.829317  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:02:08.831799 15412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:08.836196 15412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e769cd3e5b24d8b882df46ee02d1f72: Bootstrap starting.
I20260812 08:02:08.838493 15412 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e769cd3e5b24d8b882df46ee02d1f72: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:08.839195 15412 log.cc:826] T 00000000000000000000000000000000 P 3e769cd3e5b24d8b882df46ee02d1f72: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:08.840713 15412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e769cd3e5b24d8b882df46ee02d1f72: No bootstrap required, opened a new log
I20260812 08:02:08.843039 15412 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3e769cd3e5b24d8b882df46ee02d1f72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e769cd3e5b24d8b882df46ee02d1f72" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 35287 } }
I20260812 08:02:08.843202 15412 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3e769cd3e5b24d8b882df46ee02d1f72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:08.843250 15412 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3e769cd3e5b24d8b882df46ee02d1f72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e769cd3e5b24d8b882df46ee02d1f72, State: Initialized, Role: FOLLOWER
I20260812 08:02:08.843582 15412 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e769cd3e5b24d8b882df46ee02d1f72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e769cd3e5b24d8b882df46ee02d1f72" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 35287 } }
I20260812 08:02:08.843683 15412 raft_consensus.cc:399] T 00000000000000000000000000000000 P 3e769cd3e5b24d8b882df46ee02d1f72 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:08.843722 15412 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3e769cd3e5b24d8b882df46ee02d1f72 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:08.843792 15412 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3e769cd3e5b24d8b882df46ee02d1f72 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:08.844723 15412 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3e769cd3e5b24d8b882df46ee02d1f72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e769cd3e5b24d8b882df46ee02d1f72" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 35287 } }
I20260812 08:02:08.845044 15412 leader_election.cc:304] T 00000000000000000000000000000000 P 3e769cd3e5b24d8b882df46ee02d1f72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e769cd3e5b24d8b882df46ee02d1f72; no voters: 
I20260812 08:02:08.845284 15412 leader_election.cc:290] T 00000000000000000000000000000000 P 3e769cd3e5b24d8b882df46ee02d1f72 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:02:08.845405 15422 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3e769cd3e5b24d8b882df46ee02d1f72 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:02:08.845584 15422 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3e769cd3e5b24d8b882df46ee02d1f72 [term 1 LEADER]: Becoming Leader. State: Replica: 3e769cd3e5b24d8b882df46ee02d1f72, State: Running, Role: LEADER
I20260812 08:02:08.845877 15422 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e769cd3e5b24d8b882df46ee02d1f72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e769cd3e5b24d8b882df46ee02d1f72" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 35287 } }
I20260812 08:02:08.846266 15412 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3e769cd3e5b24d8b882df46ee02d1f72 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:02:08.847424 15426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e769cd3e5b24d8b882df46ee02d1f72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e769cd3e5b24d8b882df46ee02d1f72. Latest consensus state: current_term: 1 leader_uuid: "3e769cd3e5b24d8b882df46ee02d1f72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e769cd3e5b24d8b882df46ee02d1f72" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 35287 } } }
I20260812 08:02:08.847569 15426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e769cd3e5b24d8b882df46ee02d1f72 [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:08.847424 15424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e769cd3e5b24d8b882df46ee02d1f72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e769cd3e5b24d8b882df46ee02d1f72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e769cd3e5b24d8b882df46ee02d1f72" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 35287 } } }
I20260812 08:02:08.847999 15424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e769cd3e5b24d8b882df46ee02d1f72 [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:08.848272 15441 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:02:08.851217 15441 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:02:08.856367 15441 catalog_manager.cc:1383] Generated new cluster ID: 60afc4b0bd4d4f20a531cf5e0b439d03
I20260812 08:02:08.856496 15441 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:02:08.871927 15441 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:02:08.872826 15441 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:02:08.879531 15441 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 3e769cd3e5b24d8b882df46ee02d1f72: Generated new TSK 0
I20260812 08:02:08.880208 15441 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:02:08.894143  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.129:0
--local_ip_for_outbound_sockets=127.5.142.129
--webserver_interface=127.5.142.129
--webserver_port=0
--tserver_master_addrs=127.5.142.190:35287
--builtin_ntp_servers=127.5.142.148:34105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20260812 08:02:09.001461 15458 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:02:09.001703 15458 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:09.001739 15458 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:09.001801 15458 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:09.005125 15458 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:02:09.005180 15458 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:02:09.005213 15458 flags.cc:432] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20260812 08:02:09.005247 15458 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:09.005332 15458 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:02:09.005366 15458 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.129
I20260812 08:02:09.009232 15458 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:34105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.5.142.129
--webserver_port=0
--tserver_master_addrs=127.5.142.190:35287
--never_fsync=true
--heap_profile_path=/tmp/kudu.15458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.129

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:09.010285 15458 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:09.011407 15458 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:09.017743 15472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:09.017766 15475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:09.017868 15458 server_base.cc:1061] running on GCE node
W20260812 08:02:09.017735 15470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:09.018301 15458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:09.018803 15458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:09.019946 15458 hybrid_clock.cc:648] HybridClock initialized: now 1786521729019933 us; error 32 us; skew 500 ppm
I20260812 08:02:09.021570 15458 webserver.cc:533] Webserver started at http://127.5.142.129:43463/ using document root <none> and password file <none>
I20260812 08:02:09.022148 15458 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:09.022220 15458 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:09.022418 15458 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:09.023927 15458 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "d7e9a2fb76ed4dc0aff86f1902fd5ff0"
format_stamp: "Formatted at 2026-08-12 08:02:09 on dist-test-slave-ncp9"
I20260812 08:02:09.024403 15458 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "d7e9a2fb76ed4dc0aff86f1902fd5ff0"
format_stamp: "Formatted at 2026-08-12 08:02:09 on dist-test-slave-ncp9"
I20260812 08:02:09.027644 15458 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.003s
I20260812 08:02:09.029794 15483 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:09.030733 15458 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:09.030838 15458 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "d7e9a2fb76ed4dc0aff86f1902fd5ff0"
format_stamp: "Formatted at 2026-08-12 08:02:09 on dist-test-slave-ncp9"
I20260812 08:02:09.030923 15458 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:09.045902 15458 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:09.046551 15458 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:09.046711 15458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:09.047278 15458 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:09.048307 15458 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:09.048357 15458 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:09.048413 15458 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:09.048439 15458 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:09.057370 15458 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.129:37781
I20260812 08:02:09.057415 15682 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.129:37781 every 8 connection(s)
I20260812 08:02:09.058312 15458 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:02:09.062577  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 15458
I20260812 08:02:09.062737  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:02:09.065397  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.130:0
--local_ip_for_outbound_sockets=127.5.142.130
--webserver_interface=127.5.142.130
--webserver_port=0
--tserver_master_addrs=127.5.142.190:35287
--builtin_ntp_servers=127.5.142.148:34105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20260812 08:02:09.067559 15683 heartbeater.cc:344] Connected to a master server at 127.5.142.190:35287
I20260812 08:02:09.067843 15683 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:09.068331 15683 heartbeater.cc:507] Master 127.5.142.190:35287 requested a full tablet report, sending...
I20260812 08:02:09.069995 15343 ts_manager.cc:194] Registered new tserver with Master: d7e9a2fb76ed4dc0aff86f1902fd5ff0 (127.5.142.129:37781)
I20260812 08:02:09.071578 15343 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.129:36297
W20260812 08:02:09.182670 15691 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:02:09.182934 15691 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:09.182964 15691 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:09.183012 15691 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:09.186486 15691 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:02:09.186546 15691 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:02:09.186571 15691 flags.cc:432] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20260812 08:02:09.186595 15691 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:09.186666 15691 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:02:09.186694 15691 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.130
I20260812 08:02:09.190686 15691 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:34105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.5.142.130
--webserver_port=0
--tserver_master_addrs=127.5.142.190:35287
--never_fsync=true
--heap_profile_path=/tmp/kudu.15691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.130

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:09.191802 15691 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:09.192993 15691 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260812 08:02:09.199975 15691 server_base.cc:1061] running on GCE node
W20260812 08:02:09.199889 15700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:09.199889 15711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:09.199892 15705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:09.200532 15691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:09.201138 15691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:09.202291 15691 hybrid_clock.cc:648] HybridClock initialized: now 1786521729202277 us; error 32 us; skew 500 ppm
I20260812 08:02:09.204046 15691 webserver.cc:533] Webserver started at http://127.5.142.130:42829/ using document root <none> and password file <none>
I20260812 08:02:09.204630 15691 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:09.204691 15691 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:09.204885 15691 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:09.206514 15691 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "a8e778da4a0c4b15bf738e5fcbf822e5"
format_stamp: "Formatted at 2026-08-12 08:02:09 on dist-test-slave-ncp9"
I20260812 08:02:09.206995 15691 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "a8e778da4a0c4b15bf738e5fcbf822e5"
format_stamp: "Formatted at 2026-08-12 08:02:09 on dist-test-slave-ncp9"
I20260812 08:02:09.210342 15691 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:02:09.212456 15721 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:09.213479 15691 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:02:09.213584 15691 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "a8e778da4a0c4b15bf738e5fcbf822e5"
format_stamp: "Formatted at 2026-08-12 08:02:09 on dist-test-slave-ncp9"
I20260812 08:02:09.213670 15691 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:09.224349 15691 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:09.225029 15691 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:09.225193 15691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:09.225775 15691 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:09.226806 15691 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:09.226855 15691 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:09.226907 15691 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:09.226936 15691 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:09.235731 15691 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.130:33695
I20260812 08:02:09.235770 15929 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.130:33695 every 8 connection(s)
I20260812 08:02:09.236729 15691 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:02:09.245043 15934 heartbeater.cc:344] Connected to a master server at 127.5.142.190:35287
I20260812 08:02:09.245275 15934 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:09.245533  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 15691
I20260812 08:02:09.245656  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:02:09.245748 15934 heartbeater.cc:507] Master 127.5.142.190:35287 requested a full tablet report, sending...
I20260812 08:02:09.246850 15342 ts_manager.cc:194] Registered new tserver with Master: a8e778da4a0c4b15bf738e5fcbf822e5 (127.5.142.130:33695)
I20260812 08:02:09.247421 15342 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.130:60715
I20260812 08:02:09.247871  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.131:0
--local_ip_for_outbound_sockets=127.5.142.131
--webserver_interface=127.5.142.131
--webserver_port=0
--tserver_master_addrs=127.5.142.190:35287
--builtin_ntp_servers=127.5.142.148:34105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20260812 08:02:09.355808 15938 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:02:09.356079 15938 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:09.356123 15938 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:09.356189 15938 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:09.359535 15938 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:02:09.359603 15938 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:02:09.359628 15938 flags.cc:432] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20260812 08:02:09.359658 15938 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:09.359741 15938 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:02:09.359774 15938 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.131
I20260812 08:02:09.363969 15938 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:34105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.5.142.131
--webserver_port=0
--tserver_master_addrs=127.5.142.190:35287
--never_fsync=true
--heap_profile_path=/tmp/kudu.15938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.131

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:09.365185 15938 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:09.366323 15938 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:09.372578 15948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:09.372622 15949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:09.372588 15952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:09.372926 15938 server_base.cc:1061] running on GCE node
I20260812 08:02:09.373272 15938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:09.373788 15938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:09.374959 15938 hybrid_clock.cc:648] HybridClock initialized: now 1786521729374934 us; error 27 us; skew 500 ppm
I20260812 08:02:09.376560 15938 webserver.cc:533] Webserver started at http://127.5.142.131:34657/ using document root <none> and password file <none>
I20260812 08:02:09.377148 15938 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:09.377210 15938 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:09.377408 15938 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:09.378916 15938 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "d743640a5d9045bcb62af8209f960f92"
format_stamp: "Formatted at 2026-08-12 08:02:09 on dist-test-slave-ncp9"
I20260812 08:02:09.379400 15938 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "d743640a5d9045bcb62af8209f960f92"
format_stamp: "Formatted at 2026-08-12 08:02:09 on dist-test-slave-ncp9"
I20260812 08:02:09.382688 15938 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:02:09.384678 15964 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:09.385649 15938 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:09.385753 15938 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "d743640a5d9045bcb62af8209f960f92"
format_stamp: "Formatted at 2026-08-12 08:02:09 on dist-test-slave-ncp9"
I20260812 08:02:09.385833 15938 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:09.410194 15938 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:09.410956 15938 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:09.411144 15938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:09.411791 15938 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:09.412881 15938 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:09.412964 15938 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:09.413029 15938 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:09.413054 15938 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:09.422741 15938 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.131:44535
I20260812 08:02:09.422772 16162 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.131:44535 every 8 connection(s)
I20260812 08:02:09.423779 15938 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:02:09.426213  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 15938
I20260812 08:02:09.426339  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:02:09.432474 16163 heartbeater.cc:344] Connected to a master server at 127.5.142.190:35287
I20260812 08:02:09.432826 16163 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:09.433281 16163 heartbeater.cc:507] Master 127.5.142.190:35287 requested a full tablet report, sending...
I20260812 08:02:09.434242 15342 ts_manager.cc:194] Registered new tserver with Master: d743640a5d9045bcb62af8209f960f92 (127.5.142.131:44535)
I20260812 08:02:09.434782 15342 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.131:36007
I20260812 08:02:09.440488  5690 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:02:09.452230 15342 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37866:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
I20260812 08:02:09.470562 15800 tablet_service.cc:1511] Processing CreateTablet for tablet bb793fc37e9c458b8650fdde330f683d (DEFAULT_TABLE table=TestTable [id=9fa4c6bb433644399ea5bb3c70307b95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:09.470989 15574 tablet_service.cc:1511] Processing CreateTablet for tablet bb793fc37e9c458b8650fdde330f683d (DEFAULT_TABLE table=TestTable [id=9fa4c6bb433644399ea5bb3c70307b95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:09.471524 15800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb793fc37e9c458b8650fdde330f683d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:09.471498 16040 tablet_service.cc:1511] Processing CreateTablet for tablet bb793fc37e9c458b8650fdde330f683d (DEFAULT_TABLE table=TestTable [id=9fa4c6bb433644399ea5bb3c70307b95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:09.471984 15574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb793fc37e9c458b8650fdde330f683d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:09.472383 16040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb793fc37e9c458b8650fdde330f683d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:09.476564 16188 tablet_bootstrap.cc:492] T bb793fc37e9c458b8650fdde330f683d P a8e778da4a0c4b15bf738e5fcbf822e5: Bootstrap starting.
I20260812 08:02:09.477825 16198 tablet_bootstrap.cc:492] T bb793fc37e9c458b8650fdde330f683d P d743640a5d9045bcb62af8209f960f92: Bootstrap starting.
I20260812 08:02:09.478407 16196 tablet_bootstrap.cc:492] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0: Bootstrap starting.
I20260812 08:02:09.479669 16188 tablet_bootstrap.cc:654] T bb793fc37e9c458b8650fdde330f683d P a8e778da4a0c4b15bf738e5fcbf822e5: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:09.479976 16198 tablet_bootstrap.cc:654] T bb793fc37e9c458b8650fdde330f683d P d743640a5d9045bcb62af8209f960f92: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:09.480418 16188 log.cc:826] T bb793fc37e9c458b8650fdde330f683d P a8e778da4a0c4b15bf738e5fcbf822e5: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:09.480561 16196 tablet_bootstrap.cc:654] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:09.480783 16198 log.cc:826] T bb793fc37e9c458b8650fdde330f683d P d743640a5d9045bcb62af8209f960f92: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:09.481348 16196 log.cc:826] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:09.482244 16188 tablet_bootstrap.cc:492] T bb793fc37e9c458b8650fdde330f683d P a8e778da4a0c4b15bf738e5fcbf822e5: No bootstrap required, opened a new log
I20260812 08:02:09.482440 16188 ts_tablet_manager.cc:1403] T bb793fc37e9c458b8650fdde330f683d P a8e778da4a0c4b15bf738e5fcbf822e5: Time spent bootstrapping tablet: real 0.006s	user 0.004s	sys 0.000s
I20260812 08:02:09.482517 16198 tablet_bootstrap.cc:492] T bb793fc37e9c458b8650fdde330f683d P d743640a5d9045bcb62af8209f960f92: No bootstrap required, opened a new log
I20260812 08:02:09.482707 16198 ts_tablet_manager.cc:1403] T bb793fc37e9c458b8650fdde330f683d P d743640a5d9045bcb62af8209f960f92: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:02:09.483007 16196 tablet_bootstrap.cc:492] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0: No bootstrap required, opened a new log
I20260812 08:02:09.483191 16196 ts_tablet_manager.cc:1403] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0: Time spent bootstrapping tablet: real 0.005s	user 0.003s	sys 0.000s
I20260812 08:02:09.485459 16188 raft_consensus.cc:359] T bb793fc37e9c458b8650fdde330f683d P a8e778da4a0c4b15bf738e5fcbf822e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8e778da4a0c4b15bf738e5fcbf822e5" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33695 } } peers { permanent_uuid: "d7e9a2fb76ed4dc0aff86f1902fd5ff0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 37781 } } peers { permanent_uuid: "d743640a5d9045bcb62af8209f960f92" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 44535 } }
I20260812 08:02:09.485637 16188 raft_consensus.cc:385] T bb793fc37e9c458b8650fdde330f683d P a8e778da4a0c4b15bf738e5fcbf822e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:09.485692 16188 raft_consensus.cc:740] T bb793fc37e9c458b8650fdde330f683d P a8e778da4a0c4b15bf738e5fcbf822e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8e778da4a0c4b15bf738e5fcbf822e5, State: Initialized, Role: FOLLOWER
I20260812 08:02:09.485716 16198 raft_consensus.cc:359] T bb793fc37e9c458b8650fdde330f683d P d743640a5d9045bcb62af8209f960f92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8e778da4a0c4b15bf738e5fcbf822e5" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33695 } } peers { permanent_uuid: "d7e9a2fb76ed4dc0aff86f1902fd5ff0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 37781 } } peers { permanent_uuid: "d743640a5d9045bcb62af8209f960f92" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 44535 } }
I20260812 08:02:09.485913 16198 raft_consensus.cc:385] T bb793fc37e9c458b8650fdde330f683d P d743640a5d9045bcb62af8209f960f92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:09.485988 16198 raft_consensus.cc:740] T bb793fc37e9c458b8650fdde330f683d P d743640a5d9045bcb62af8209f960f92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d743640a5d9045bcb62af8209f960f92, State: Initialized, Role: FOLLOWER
I20260812 08:02:09.486140 16188 consensus_queue.cc:260] T bb793fc37e9c458b8650fdde330f683d P a8e778da4a0c4b15bf738e5fcbf822e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8e778da4a0c4b15bf738e5fcbf822e5" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33695 } } peers { permanent_uuid: "d7e9a2fb76ed4dc0aff86f1902fd5ff0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 37781 } } peers { permanent_uuid: "d743640a5d9045bcb62af8209f960f92" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 44535 } }
I20260812 08:02:09.486167 16196 raft_consensus.cc:359] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8e778da4a0c4b15bf738e5fcbf822e5" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33695 } } peers { permanent_uuid: "d7e9a2fb76ed4dc0aff86f1902fd5ff0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 37781 } } peers { permanent_uuid: "d743640a5d9045bcb62af8209f960f92" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 44535 } }
I20260812 08:02:09.486351 16196 raft_consensus.cc:385] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:09.486404 16196 raft_consensus.cc:740] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e9a2fb76ed4dc0aff86f1902fd5ff0, State: Initialized, Role: FOLLOWER
I20260812 08:02:09.486416 16198 consensus_queue.cc:260] T bb793fc37e9c458b8650fdde330f683d P d743640a5d9045bcb62af8209f960f92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8e778da4a0c4b15bf738e5fcbf822e5" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33695 } } peers { permanent_uuid: "d7e9a2fb76ed4dc0aff86f1902fd5ff0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 37781 } } peers { permanent_uuid: "d743640a5d9045bcb62af8209f960f92" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 44535 } }
I20260812 08:02:09.486896 15934 heartbeater.cc:499] Master 127.5.142.190:35287 was elected leader, sending a full tablet report...
I20260812 08:02:09.486869 16196 consensus_queue.cc:260] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8e778da4a0c4b15bf738e5fcbf822e5" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33695 } } peers { permanent_uuid: "d7e9a2fb76ed4dc0aff86f1902fd5ff0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 37781 } } peers { permanent_uuid: "d743640a5d9045bcb62af8209f960f92" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 44535 } }
I20260812 08:02:09.487047 16188 ts_tablet_manager.cc:1434] T bb793fc37e9c458b8650fdde330f683d P a8e778da4a0c4b15bf738e5fcbf822e5: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260812 08:02:09.487247 16198 ts_tablet_manager.cc:1434] T bb793fc37e9c458b8650fdde330f683d P d743640a5d9045bcb62af8209f960f92: Time spent starting tablet: real 0.004s	user 0.006s	sys 0.000s
I20260812 08:02:09.487579 16163 heartbeater.cc:499] Master 127.5.142.190:35287 was elected leader, sending a full tablet report...
I20260812 08:02:09.487618 16196 ts_tablet_manager.cc:1434] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:02:09.487915 15683 heartbeater.cc:499] Master 127.5.142.190:35287 was elected leader, sending a full tablet report...
W20260812 08:02:09.524696 16165 tablet.cc:2406] T bb793fc37e9c458b8650fdde330f683d P d743640a5d9045bcb62af8209f960f92: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:02:09.538033 15935 tablet.cc:2406] T bb793fc37e9c458b8650fdde330f683d P a8e778da4a0c4b15bf738e5fcbf822e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:02:09.559968 15687 tablet.cc:2406] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:02:09.623270 16205 raft_consensus.cc:493] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:02:09.623505 16205 raft_consensus.cc:515] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8e778da4a0c4b15bf738e5fcbf822e5" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33695 } } peers { permanent_uuid: "d7e9a2fb76ed4dc0aff86f1902fd5ff0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 37781 } } peers { permanent_uuid: "d743640a5d9045bcb62af8209f960f92" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 44535 } }
I20260812 08:02:09.624655 16205 leader_election.cc:290] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8e778da4a0c4b15bf738e5fcbf822e5 (127.5.142.130:33695), d743640a5d9045bcb62af8209f960f92 (127.5.142.131:44535)
I20260812 08:02:09.629345 15849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb793fc37e9c458b8650fdde330f683d" candidate_uuid: "d7e9a2fb76ed4dc0aff86f1902fd5ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8e778da4a0c4b15bf738e5fcbf822e5" is_pre_election: true
I20260812 08:02:09.629657 15849 raft_consensus.cc:2468] T bb793fc37e9c458b8650fdde330f683d P a8e778da4a0c4b15bf738e5fcbf822e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e9a2fb76ed4dc0aff86f1902fd5ff0 in term 0.
I20260812 08:02:09.629644 16084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb793fc37e9c458b8650fdde330f683d" candidate_uuid: "d7e9a2fb76ed4dc0aff86f1902fd5ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d743640a5d9045bcb62af8209f960f92" is_pre_election: true
I20260812 08:02:09.629882 16084 raft_consensus.cc:2468] T bb793fc37e9c458b8650fdde330f683d P d743640a5d9045bcb62af8209f960f92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e9a2fb76ed4dc0aff86f1902fd5ff0 in term 0.
I20260812 08:02:09.630177 15486 leader_election.cc:304] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8e778da4a0c4b15bf738e5fcbf822e5, d7e9a2fb76ed4dc0aff86f1902fd5ff0; no voters: 
I20260812 08:02:09.630434 16205 raft_consensus.cc:2804] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:02:09.630523 16205 raft_consensus.cc:493] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:02:09.630584 16205 raft_consensus.cc:3060] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:09.631524 16205 raft_consensus.cc:515] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8e778da4a0c4b15bf738e5fcbf822e5" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33695 } } peers { permanent_uuid: "d7e9a2fb76ed4dc0aff86f1902fd5ff0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 37781 } } peers { permanent_uuid: "d743640a5d9045bcb62af8209f960f92" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 44535 } }
I20260812 08:02:09.631901 16205 leader_election.cc:290] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0 [CANDIDATE]: Term 1 election: Requested vote from peers a8e778da4a0c4b15bf738e5fcbf822e5 (127.5.142.130:33695), d743640a5d9045bcb62af8209f960f92 (127.5.142.131:44535)
I20260812 08:02:09.632304 15849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb793fc37e9c458b8650fdde330f683d" candidate_uuid: "d7e9a2fb76ed4dc0aff86f1902fd5ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8e778da4a0c4b15bf738e5fcbf822e5"
I20260812 08:02:09.632381 16084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb793fc37e9c458b8650fdde330f683d" candidate_uuid: "d7e9a2fb76ed4dc0aff86f1902fd5ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d743640a5d9045bcb62af8209f960f92"
I20260812 08:02:09.632445 15849 raft_consensus.cc:3060] T bb793fc37e9c458b8650fdde330f683d P a8e778da4a0c4b15bf738e5fcbf822e5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:09.632493 16084 raft_consensus.cc:3060] T bb793fc37e9c458b8650fdde330f683d P d743640a5d9045bcb62af8209f960f92 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:09.633566 15849 raft_consensus.cc:2468] T bb793fc37e9c458b8650fdde330f683d P a8e778da4a0c4b15bf738e5fcbf822e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e9a2fb76ed4dc0aff86f1902fd5ff0 in term 1.
I20260812 08:02:09.633566 16084 raft_consensus.cc:2468] T bb793fc37e9c458b8650fdde330f683d P d743640a5d9045bcb62af8209f960f92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e9a2fb76ed4dc0aff86f1902fd5ff0 in term 1.
I20260812 08:02:09.633899 15486 leader_election.cc:304] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d743640a5d9045bcb62af8209f960f92, d7e9a2fb76ed4dc0aff86f1902fd5ff0; no voters: 
I20260812 08:02:09.634085 16205 raft_consensus.cc:2804] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:02:09.634347 16205 raft_consensus.cc:697] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0 [term 1 LEADER]: Becoming Leader. State: Replica: d7e9a2fb76ed4dc0aff86f1902fd5ff0, State: Running, Role: LEADER
I20260812 08:02:09.634708 16205 consensus_queue.cc:237] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8e778da4a0c4b15bf738e5fcbf822e5" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33695 } } peers { permanent_uuid: "d7e9a2fb76ed4dc0aff86f1902fd5ff0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 37781 } } peers { permanent_uuid: "d743640a5d9045bcb62af8209f960f92" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 44535 } }
I20260812 08:02:09.637920 15338 catalog_manager.cc:5719] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e9a2fb76ed4dc0aff86f1902fd5ff0 (127.5.142.129). New cstate: current_term: 1 leader_uuid: "d7e9a2fb76ed4dc0aff86f1902fd5ff0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8e778da4a0c4b15bf738e5fcbf822e5" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33695 } } peers { permanent_uuid: "d7e9a2fb76ed4dc0aff86f1902fd5ff0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 37781 } } peers { permanent_uuid: "d743640a5d9045bcb62af8209f960f92" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 44535 } } }
I20260812 08:02:09.653813  5690 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:02:09.655392  5690 ts_itest-base.cc:246] Waiting for 1 tablets on tserver d7e9a2fb76ed4dc0aff86f1902fd5ff0 to finish bootstrapping
I20260812 08:02:09.660863  5690 ts_itest-base.cc:246] Waiting for 1 tablets on tserver a8e778da4a0c4b15bf738e5fcbf822e5 to finish bootstrapping
I20260812 08:02:09.665591  5690 ts_itest-base.cc:246] Waiting for 1 tablets on tserver d743640a5d9045bcb62af8209f960f92 to finish bootstrapping
I20260812 08:02:09.681814  5690 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 0.0 1.1
I20260812 08:02:09.784614  5690 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 0.0 1.1
I20260812 08:02:09.987274  5690 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 0.0 1.1
I20260812 08:02:10.063683 16205 consensus_queue.cc:1048] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d743640a5d9045bcb62af8209f960f92" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 44535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:02:10.069077 16205 consensus_queue.cc:1048] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8e778da4a0c4b15bf738e5fcbf822e5" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:02:10.312841  5690 test_workload.cc:405] TestWorkload: Skipping table creation because table TestTable already exists
I20260812 08:02:10.313158  5690 raft_consensus-itest-base.cc:268] Waiting until we've written at least 4MB...
I20260812 08:02:10.363379 15687 maintenance_manager.cc:419] P d7e9a2fb76ed4dc0aff86f1902fd5ff0: Scheduling FlushMRSOp(bb793fc37e9c458b8650fdde330f683d): 1840834 bytes log retention, and flush 1958153 bytes memory
I20260812 08:02:10.404374 15494 maintenance_manager.cc:643] P d7e9a2fb76ed4dc0aff86f1902fd5ff0: FlushMRSOp(bb793fc37e9c458b8650fdde330f683d) complete. Timing: real 0.041s	user 0.033s	sys 0.000s Metrics: {"bytes_written":135803,"cfile_init":1,"compiler_manager_pool.queue_time_us":216,"dirs.queue_time_us":240,"dirs.run_cpu_time_us":195,"dirs.run_wall_time_us":731,"drs_written":1,"lbm_read_time_us":77,"lbm_reads_lt_1ms":4,"lbm_write_time_us":777,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":13,"thread_start_us":270,"threads_started":2}
I20260812 08:02:10.405654 15687 maintenance_manager.cc:419] P d7e9a2fb76ed4dc0aff86f1902fd5ff0: Scheduling LogGCOp(bb793fc37e9c458b8650fdde330f683d): free 920416 bytes of WAL
I20260812 08:02:10.405963 15494 log_reader.cc:385] T bb793fc37e9c458b8650fdde330f683d: removed 1 log segments from log reader
I20260812 08:02:10.406049 15494 log.cc:1079] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0: Deleting log segment in path: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal/wals/bb793fc37e9c458b8650fdde330f683d/wal-000000001 (ops 1-8)
I20260812 08:02:10.406500 15494 maintenance_manager.cc:643] P d7e9a2fb76ed4dc0aff86f1902fd5ff0: LogGCOp(bb793fc37e9c458b8650fdde330f683d) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20260812 08:02:10.406778 15687 maintenance_manager.cc:419] P d7e9a2fb76ed4dc0aff86f1902fd5ff0: Scheduling FlushMRSOp(bb793fc37e9c458b8650fdde330f683d): 2761330 bytes log retention, and flush 2998537 bytes memory
I20260812 08:02:10.432591 15494 maintenance_manager.cc:643] P d7e9a2fb76ed4dc0aff86f1902fd5ff0: FlushMRSOp(bb793fc37e9c458b8650fdde330f683d) complete. Timing: real 0.026s	user 0.022s	sys 0.000s Metrics: {"bytes_written":135803,"cfile_init":1,"dirs.queue_time_us":1602,"dirs.run_cpu_time_us":198,"dirs.run_wall_time_us":1246,"drs_written":1,"lbm_read_time_us":57,"lbm_reads_lt_1ms":4,"lbm_write_time_us":645,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":13}
I20260812 08:02:10.433748 15687 maintenance_manager.cc:419] P d7e9a2fb76ed4dc0aff86f1902fd5ff0: Scheduling LogGCOp(bb793fc37e9c458b8650fdde330f683d): free 1840912 bytes of WAL
I20260812 08:02:10.433949 15494 log_reader.cc:385] T bb793fc37e9c458b8650fdde330f683d: removed 2 log segments from log reader
I20260812 08:02:10.434000 15494 log.cc:1079] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0: Deleting log segment in path: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal/wals/bb793fc37e9c458b8650fdde330f683d/wal-000000002 (ops 9-15)
I20260812 08:02:10.434056 15494 log.cc:1079] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0: Deleting log segment in path: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal/wals/bb793fc37e9c458b8650fdde330f683d/wal-000000003 (ops 16-22)
I20260812 08:02:10.434530  5690 raft_consensus-itest-base.cc:274] Waiting for log GC on d7e9a2fb76ed4dc0aff86f1902fd5ff0
I20260812 08:02:10.434765  5690 raft_consensus-itest-base.cc:279] Log GC complete on d7e9a2fb76ed4dc0aff86f1902fd5ff0
I20260812 08:02:10.435189 15494 maintenance_manager.cc:643] P d7e9a2fb76ed4dc0aff86f1902fd5ff0: LogGCOp(bb793fc37e9c458b8650fdde330f683d) complete. Timing: real 0.001s	user 0.001s	sys 0.000s Metrics: {}
I20260812 08:02:10.435514 15687 maintenance_manager.cc:419] P d7e9a2fb76ed4dc0aff86f1902fd5ff0: Scheduling FlushMRSOp(bb793fc37e9c458b8650fdde330f683d): 1840912 bytes log retention, and flush 1958153 bytes memory
I20260812 08:02:10.441679 15935 maintenance_manager.cc:419] P a8e778da4a0c4b15bf738e5fcbf822e5: Scheduling FlushMRSOp(bb793fc37e9c458b8650fdde330f683d): 4339134 bytes log retention, and flush 6119689 bytes memory
I20260812 08:02:10.456108 15494 maintenance_manager.cc:643] P d7e9a2fb76ed4dc0aff86f1902fd5ff0: FlushMRSOp(bb793fc37e9c458b8650fdde330f683d) complete. Timing: real 0.020s	user 0.018s	sys 0.000s Metrics: {"bytes_written":135803,"cfile_init":1,"dirs.queue_time_us":52,"dirs.run_cpu_time_us":147,"dirs.run_wall_time_us":1382,"drs_written":1,"lbm_read_time_us":60,"lbm_reads_lt_1ms":4,"lbm_write_time_us":731,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":11}
I20260812 08:02:10.456815 15687 maintenance_manager.cc:419] P d7e9a2fb76ed4dc0aff86f1902fd5ff0: Scheduling UndoDeltaBlockGCOp(bb793fc37e9c458b8650fdde330f683d): 844 bytes on disk
I20260812 08:02:10.457880 15494 maintenance_manager.cc:643] P d7e9a2fb76ed4dc0aff86f1902fd5ff0: UndoDeltaBlockGCOp(bb793fc37e9c458b8650fdde330f683d) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {"cfile_init":3,"lbm_read_time_us":174,"lbm_reads_lt_1ms":12}
I20260812 08:02:10.458472 15687 maintenance_manager.cc:419] P d7e9a2fb76ed4dc0aff86f1902fd5ff0: Scheduling CompactRowSetsOp(bb793fc37e9c458b8650fdde330f683d): perf score=1.797296
I20260812 08:02:10.507469 15727 maintenance_manager.cc:643] P a8e778da4a0c4b15bf738e5fcbf822e5: FlushMRSOp(bb793fc37e9c458b8650fdde330f683d) complete. Timing: real 0.065s	user 0.052s	sys 0.009s Metrics: {"bytes_written":136001,"cfile_init":1,"compiler_manager_pool.queue_time_us":1356,"drs_written":1,"lbm_read_time_us":74,"lbm_reads_lt_1ms":4,"lbm_write_time_us":811,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":37,"thread_start_us":295,"threads_started":2}
I20260812 08:02:10.508541 15935 maintenance_manager.cc:419] P a8e778da4a0c4b15bf738e5fcbf822e5: Scheduling LogGCOp(bb793fc37e9c458b8650fdde330f683d): free 2629751 bytes of WAL
I20260812 08:02:10.508816 15727 log_reader.cc:385] T bb793fc37e9c458b8650fdde330f683d: removed 3 log segments from log reader
I20260812 08:02:10.508886 15727 log.cc:1079] T bb793fc37e9c458b8650fdde330f683d P a8e778da4a0c4b15bf738e5fcbf822e5: Deleting log segment in path: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal/wals/bb793fc37e9c458b8650fdde330f683d/wal-000000001 (ops 1-8)
I20260812 08:02:10.508980 15727 log.cc:1079] T bb793fc37e9c458b8650fdde330f683d P a8e778da4a0c4b15bf738e5fcbf822e5: Deleting log segment in path: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal/wals/bb793fc37e9c458b8650fdde330f683d/wal-000000002 (ops 9-14)
I20260812 08:02:10.509013 15727 log.cc:1079] T bb793fc37e9c458b8650fdde330f683d P a8e778da4a0c4b15bf738e5fcbf822e5: Deleting log segment in path: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal/wals/bb793fc37e9c458b8650fdde330f683d/wal-000000003 (ops 15-21)
I20260812 08:02:10.510028 15727 maintenance_manager.cc:643] P a8e778da4a0c4b15bf738e5fcbf822e5: LogGCOp(bb793fc37e9c458b8650fdde330f683d) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20260812 08:02:10.510358 15935 maintenance_manager.cc:419] P a8e778da4a0c4b15bf738e5fcbf822e5: Scheduling UndoDeltaBlockGCOp(bb793fc37e9c458b8650fdde330f683d): 558 bytes on disk
I20260812 08:02:10.511296 15727 maintenance_manager.cc:643] P a8e778da4a0c4b15bf738e5fcbf822e5: UndoDeltaBlockGCOp(bb793fc37e9c458b8650fdde330f683d) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":65,"lbm_reads_lt_1ms":4}
I20260812 08:02:10.523116 15494 maintenance_manager.cc:643] P d7e9a2fb76ed4dc0aff86f1902fd5ff0: CompactRowSetsOp(bb793fc37e9c458b8650fdde330f683d) complete. Timing: real 0.064s	user 0.064s	sys 0.000s Metrics: {"bytes_written":136001,"cfile_cache_hit":15,"cfile_cache_hit_bytes":393588,"cfile_cache_miss":39,"cfile_cache_miss_bytes":395906,"cfile_init":7,"delta_iterators_relevant":6,"dirs.queue_time_us":1291,"dirs.run_cpu_time_us":176,"dirs.run_wall_time_us":1073,"drs_written":1,"lbm_read_time_us":1317,"lbm_reads_lt_1ms":67,"lbm_write_time_us":634,"lbm_writes_lt_1ms":23,"num_input_rowsets":3,"peak_mem_usage":98304,"rows_written":37,"thread_start_us":412,"threads_started":6}
I20260812 08:02:10.523756 15687 maintenance_manager.cc:419] P d7e9a2fb76ed4dc0aff86f1902fd5ff0: Scheduling UndoDeltaBlockGCOp(bb793fc37e9c458b8650fdde330f683d): 558 bytes on disk
I20260812 08:02:10.524187 15494 maintenance_manager.cc:643] P d7e9a2fb76ed4dc0aff86f1902fd5ff0: UndoDeltaBlockGCOp(bb793fc37e9c458b8650fdde330f683d) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":60,"lbm_reads_lt_1ms":4}
I20260812 08:02:12.436102  5690 raft_consensus-itest-base.cc:295] Servers converged with original term 1
I20260812 08:02:12.436177  5690 raft_consensus-itest-base.cc:300] Resuming d743640a5d9045bcb62af8209f960f92
I20260812 08:02:12.437026 16213 raft_consensus.cc:493] T bb793fc37e9c458b8650fdde330f683d P d743640a5d9045bcb62af8209f960f92 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d7e9a2fb76ed4dc0aff86f1902fd5ff0)
I20260812 08:02:12.437172 16213 raft_consensus.cc:515] T bb793fc37e9c458b8650fdde330f683d P d743640a5d9045bcb62af8209f960f92 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8e778da4a0c4b15bf738e5fcbf822e5" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33695 } } peers { permanent_uuid: "d7e9a2fb76ed4dc0aff86f1902fd5ff0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 37781 } } peers { permanent_uuid: "d743640a5d9045bcb62af8209f960f92" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 44535 } }
I20260812 08:02:12.438462 16213 leader_election.cc:290] T bb793fc37e9c458b8650fdde330f683d P d743640a5d9045bcb62af8209f960f92 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a8e778da4a0c4b15bf738e5fcbf822e5 (127.5.142.130:33695), d7e9a2fb76ed4dc0aff86f1902fd5ff0 (127.5.142.129:37781)
I20260812 08:02:12.441830 16281 consensus_queue.cc:753] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0 [LEADER]: The logs necessary to catch up peer d743640a5d9045bcb62af8209f960f92 have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 3..32: Segment 1 which contained index 3 has been GCed)
I20260812 08:02:12.442175 16281 raft_consensus.cc:1012] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0: Attempting to remove follower d743640a5d9045bcb62af8209f960f92 from the Raft config. Reason: The logs necessary to catch up peer d743640a5d9045bcb62af8209f960f92 have been garbage collected. The replica will never be able to catch up
I20260812 08:02:12.443009 16281 consensus_queue.cc:237] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 38, Committed index: 38, Last appended: 1.38, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "a8e778da4a0c4b15bf738e5fcbf822e5" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33695 } } peers { permanent_uuid: "d7e9a2fb76ed4dc0aff86f1902fd5ff0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 37781 } }
I20260812 08:02:12.445014 15615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb793fc37e9c458b8650fdde330f683d" candidate_uuid: "d743640a5d9045bcb62af8209f960f92" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d7e9a2fb76ed4dc0aff86f1902fd5ff0" is_pre_election: true
I20260812 08:02:12.445734 15837 raft_consensus.cc:1275] T bb793fc37e9c458b8650fdde330f683d P a8e778da4a0c4b15bf738e5fcbf822e5 [term 1 FOLLOWER]: Refusing update from remote peer d7e9a2fb76ed4dc0aff86f1902fd5ff0: Log matching property violated. Preceding OpId in replica: term: 1 index: 38. Preceding OpId from leader: term: 1 index: 39. (index mismatch)
I20260812 08:02:12.446161 16281 consensus_queue.cc:1048] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8e778da4a0c4b15bf738e5fcbf822e5" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 39, Last known committed idx: 38, Time since last communication: 0.000s
I20260812 08:02:12.448446 15837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb793fc37e9c458b8650fdde330f683d" candidate_uuid: "d743640a5d9045bcb62af8209f960f92" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a8e778da4a0c4b15bf738e5fcbf822e5" is_pre_election: true
I20260812 08:02:12.448920 15967 leader_election.cc:304] T bb793fc37e9c458b8650fdde330f683d P d743640a5d9045bcb62af8209f960f92 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d743640a5d9045bcb62af8209f960f92; no voters: a8e778da4a0c4b15bf738e5fcbf822e5, d7e9a2fb76ed4dc0aff86f1902fd5ff0
I20260812 08:02:12.449216 16213 raft_consensus.cc:2749] T bb793fc37e9c458b8650fdde330f683d P d743640a5d9045bcb62af8209f960f92 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 08:02:12.449011 16281 raft_consensus.cc:2955] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0 [term 1 LEADER]: Committing config change with OpId 1.39: config changed from index -1 to 39, VOTER d743640a5d9045bcb62af8209f960f92 (127.5.142.131) evicted. New config: { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "a8e778da4a0c4b15bf738e5fcbf822e5" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33695 } } peers { permanent_uuid: "d7e9a2fb76ed4dc0aff86f1902fd5ff0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 37781 } } }
I20260812 08:02:12.449692 15837 raft_consensus.cc:2955] T bb793fc37e9c458b8650fdde330f683d P a8e778da4a0c4b15bf738e5fcbf822e5 [term 1 FOLLOWER]: Committing config change with OpId 1.39: config changed from index -1 to 39, VOTER d743640a5d9045bcb62af8209f960f92 (127.5.142.131) evicted. New config: { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "a8e778da4a0c4b15bf738e5fcbf822e5" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33695 } } peers { permanent_uuid: "d7e9a2fb76ed4dc0aff86f1902fd5ff0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 37781 } } }
I20260812 08:02:12.452219 15343 catalog_manager.cc:5719] T bb793fc37e9c458b8650fdde330f683d P d7e9a2fb76ed4dc0aff86f1902fd5ff0 reported cstate change: config changed from index -1 to 39, VOTER d743640a5d9045bcb62af8209f960f92 (127.5.142.131) evicted. New cstate: current_term: 1 leader_uuid: "d7e9a2fb76ed4dc0aff86f1902fd5ff0" committed_config { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "a8e778da4a0c4b15bf738e5fcbf822e5" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33695 } } peers { permanent_uuid: "d7e9a2fb76ed4dc0aff86f1902fd5ff0" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 37781 } } }
I20260812 08:02:12.453307  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 15458
I20260812 08:02:12.468885 16040 tablet_service.cc:1558] Processing DeleteTablet for tablet bb793fc37e9c458b8650fdde330f683d with delete_type TABLET_DATA_TOMBSTONED (TS d743640a5d9045bcb62af8209f960f92 not found in new config with opid_index 39) from {username='slave'} at 127.0.0.1:50624
I20260812 08:02:12.469501 16308 tablet_replica.cc:333] T bb793fc37e9c458b8650fdde330f683d P d743640a5d9045bcb62af8209f960f92: stopping tablet replica
I20260812 08:02:12.469821 16308 raft_consensus.cc:2243] T bb793fc37e9c458b8650fdde330f683d P d743640a5d9045bcb62af8209f960f92 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 08:02:12.470021 16308 pending_rounds.cc:70] T bb793fc37e9c458b8650fdde330f683d P d743640a5d9045bcb62af8209f960f92: Trying to abort 1 pending ops.
I20260812 08:02:12.470088 16308 pending_rounds.cc:77] T bb793fc37e9c458b8650fdde330f683d P d743640a5d9045bcb62af8209f960f92: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7317593007415635968 op_type: WRITE_OP write_request { tablet_id: "bb793fc37e9c458b8650fdde330f683d" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\000-\336Z&1\374\272\346\000\000\000\000\000\000\000\000\000\000\002\000\000\000\000\000" indirect_data: "<redacted>""000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
I20260812 08:02:12.472925 16308 raft_consensus.cc:2272] T bb793fc37e9c458b8650fdde330f683d P d743640a5d9045bcb62af8209f960f92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 08:02:12.474225 16308 ts_tablet_manager.cc:1916] T bb793fc37e9c458b8650fdde330f683d P d743640a5d9045bcb62af8209f960f92: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 08:02:12.477334 16308 ts_tablet_manager.cc:1929] T bb793fc37e9c458b8650fdde330f683d P d743640a5d9045bcb62af8209f960f92: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20260812 08:02:12.477466 16308 log.cc:1199] T bb793fc37e9c458b8650fdde330f683d P d743640a5d9045bcb62af8209f960f92: Deleting WAL directory at /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal/wals/bb793fc37e9c458b8650fdde330f683d
I20260812 08:02:12.478341 15321 catalog_manager.cc:5050] TS d743640a5d9045bcb62af8209f960f92 (127.5.142.131:44535): tablet bb793fc37e9c458b8650fdde330f683d (table TestTable [id=9fa4c6bb433644399ea5bb3c70307b95]) successfully deleted
W20260812 08:02:12.478566 15321 connection.cc:573] server connection from 127.5.142.129:36297 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 08:02:12.478705  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 15691
I20260812 08:02:12.498123  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 15938
I20260812 08:02:12.516178  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 15295
2026-08-12T08:02:12Z chronyd exiting
[       OK ] RaftConsensusITest.TestEvictAbandonedFollowers (3891 ms)
[ RUN      ] RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled
I20260812 08:02:12.551342  5690 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:02:12.551393  5690 ts_itest-base.cc:116] --------------
I20260812 08:02:12.551407  5690 ts_itest-base.cc:117] 3 tablet servers
I20260812 08:02:12.551419  5690 ts_itest-base.cc:118] 3 replicas per TS
I20260812 08:02:12.551432  5690 ts_itest-base.cc:119] --------------
2026-08-12T08:02:12Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:02:12Z Disabled control of system clock
I20260812 08:02:12.563337  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.142.190:33763
--webserver_interface=127.5.142.190
--webserver_port=0
--builtin_ntp_servers=127.5.142.148:37867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.142.190:33763
--catalog_manager_wait_for_new_tablets_to_elect_leader=false with env {}
W20260812 08:02:12.671118 16321 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:12.671324 16321 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:12.671376 16321 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:12.674613 16321 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:02:12.674667 16321 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:12.674693 16321 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:02:12.674717 16321 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:02:12.678725 16321 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:37867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
--catalog_manager_wait_for_new_tablets_to_elect_leader=false
--ipki_ca_key_size=768
--master_addresses=127.5.142.190:33763
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.142.190:33763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.5.142.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.16321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:12.680060 16321 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:12.681399 16321 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:12.686630 16332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:12.686692 16336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:12.686625 16330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:12.686997 16321 server_base.cc:1061] running on GCE node
I20260812 08:02:12.687399 16321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:12.688220 16321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:12.689360 16321 hybrid_clock.cc:648] HybridClock initialized: now 1786521732689344 us; error 29 us; skew 500 ppm
I20260812 08:02:12.690948 16321 webserver.cc:533] Webserver started at http://127.5.142.190:44531/ using document root <none> and password file <none>
I20260812 08:02:12.691491 16321 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:12.691546 16321 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:12.691721 16321 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:12.693276 16321 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "bd8f1681ffbb4e6193807d67da6ad568"
format_stamp: "Formatted at 2026-08-12 08:02:12 on dist-test-slave-ncp9"
I20260812 08:02:12.693702 16321 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "bd8f1681ffbb4e6193807d67da6ad568"
format_stamp: "Formatted at 2026-08-12 08:02:12 on dist-test-slave-ncp9"
I20260812 08:02:12.696745 16321 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:02:12.698803 16349 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:12.699752 16321 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:12.699868 16321 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
uuid: "bd8f1681ffbb4e6193807d67da6ad568"
format_stamp: "Formatted at 2026-08-12 08:02:12 on dist-test-slave-ncp9"
I20260812 08:02:12.699945 16321 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:12.718631 16321 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:12.719379 16321 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:12.719547 16321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:12.727032 16442 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.190:33763 every 8 connection(s)
I20260812 08:02:12.727049 16321 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.190:33763
I20260812 08:02:12.728212 16321 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:02:12.729624  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 16321
I20260812 08:02:12.729743  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:02:12.731469 16445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:12.737062 16445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd8f1681ffbb4e6193807d67da6ad568: Bootstrap starting.
I20260812 08:02:12.739388 16445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd8f1681ffbb4e6193807d67da6ad568: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:12.740120 16445 log.cc:826] T 00000000000000000000000000000000 P bd8f1681ffbb4e6193807d67da6ad568: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:12.742079 16445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd8f1681ffbb4e6193807d67da6ad568: No bootstrap required, opened a new log
I20260812 08:02:12.744983 16445 raft_consensus.cc:359] T 00000000000000000000000000000000 P bd8f1681ffbb4e6193807d67da6ad568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd8f1681ffbb4e6193807d67da6ad568" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 33763 } }
I20260812 08:02:12.745170 16445 raft_consensus.cc:385] T 00000000000000000000000000000000 P bd8f1681ffbb4e6193807d67da6ad568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:12.745226 16445 raft_consensus.cc:740] T 00000000000000000000000000000000 P bd8f1681ffbb4e6193807d67da6ad568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8f1681ffbb4e6193807d67da6ad568, State: Initialized, Role: FOLLOWER
I20260812 08:02:12.745643 16445 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd8f1681ffbb4e6193807d67da6ad568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd8f1681ffbb4e6193807d67da6ad568" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 33763 } }
I20260812 08:02:12.745769 16445 raft_consensus.cc:399] T 00000000000000000000000000000000 P bd8f1681ffbb4e6193807d67da6ad568 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:12.745832 16445 raft_consensus.cc:493] T 00000000000000000000000000000000 P bd8f1681ffbb4e6193807d67da6ad568 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:12.745941 16445 raft_consensus.cc:3060] T 00000000000000000000000000000000 P bd8f1681ffbb4e6193807d67da6ad568 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:12.746769 16445 raft_consensus.cc:515] T 00000000000000000000000000000000 P bd8f1681ffbb4e6193807d67da6ad568 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd8f1681ffbb4e6193807d67da6ad568" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 33763 } }
I20260812 08:02:12.747099 16445 leader_election.cc:304] T 00000000000000000000000000000000 P bd8f1681ffbb4e6193807d67da6ad568 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd8f1681ffbb4e6193807d67da6ad568; no voters: 
I20260812 08:02:12.747383 16445 leader_election.cc:290] T 00000000000000000000000000000000 P bd8f1681ffbb4e6193807d67da6ad568 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:02:12.747502 16452 raft_consensus.cc:2804] T 00000000000000000000000000000000 P bd8f1681ffbb4e6193807d67da6ad568 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:02:12.747740 16452 raft_consensus.cc:697] T 00000000000000000000000000000000 P bd8f1681ffbb4e6193807d67da6ad568 [term 1 LEADER]: Becoming Leader. State: Replica: bd8f1681ffbb4e6193807d67da6ad568, State: Running, Role: LEADER
I20260812 08:02:12.748045 16452 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd8f1681ffbb4e6193807d67da6ad568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd8f1681ffbb4e6193807d67da6ad568" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 33763 } }
I20260812 08:02:12.748548 16445 sys_catalog.cc:565] T 00000000000000000000000000000000 P bd8f1681ffbb4e6193807d67da6ad568 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:02:12.749655 16454 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd8f1681ffbb4e6193807d67da6ad568 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd8f1681ffbb4e6193807d67da6ad568" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd8f1681ffbb4e6193807d67da6ad568" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 33763 } } }
I20260812 08:02:12.749801 16454 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd8f1681ffbb4e6193807d67da6ad568 [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:12.750052 16455 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd8f1681ffbb4e6193807d67da6ad568 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd8f1681ffbb4e6193807d67da6ad568. Latest consensus state: current_term: 1 leader_uuid: "bd8f1681ffbb4e6193807d67da6ad568" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd8f1681ffbb4e6193807d67da6ad568" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 33763 } } }
I20260812 08:02:12.750149 16455 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd8f1681ffbb4e6193807d67da6ad568 [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:12.750461 16471 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:02:12.753235 16471 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:02:12.758435 16471 catalog_manager.cc:1383] Generated new cluster ID: 84a1bf247c094cdf9c44e0964eb86786
I20260812 08:02:12.758597 16471 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:02:12.766695 16471 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:02:12.767815 16471 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:02:12.775645 16471 catalog_manager.cc:6092] T 00000000000000000000000000000000 P bd8f1681ffbb4e6193807d67da6ad568: Generated new TSK 0
I20260812 08:02:12.776283 16471 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:02:12.784469  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.129:0
--local_ip_for_outbound_sockets=127.5.142.129
--webserver_interface=127.5.142.129
--webserver_port=0
--tserver_master_addrs=127.5.142.190:33763
--builtin_ntp_servers=127.5.142.148:37867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--enable_leader_failure_detection=false with env {}
W20260812 08:02:12.894542 16489 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 08:02:12.894769 16489 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:12.894802 16489 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:12.894865 16489 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:12.898195 16489 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:12.898311 16489 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.129
I20260812 08:02:12.902065 16489 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:37867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.5.142.129
--webserver_port=0
--tserver_master_addrs=127.5.142.190:33763
--never_fsync=true
--heap_profile_path=/tmp/kudu.16489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.129

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:12.903133 16489 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:12.904307 16489 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:02:12.907111 16489 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:02:12.910605 16499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:12.910645 16498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:12.910720 16489 server_base.cc:1061] running on GCE node
W20260812 08:02:12.910768 16502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:12.911207 16489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:12.911747 16489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:12.912884 16489 hybrid_clock.cc:648] HybridClock initialized: now 1786521732912873 us; error 26 us; skew 500 ppm
I20260812 08:02:12.914485 16489 webserver.cc:533] Webserver started at http://127.5.142.129:46323/ using document root <none> and password file <none>
I20260812 08:02:12.915035 16489 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:12.915088 16489 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:12.915272 16489 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:12.916764 16489 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "667d51c376fb4f57bac5ccd43eb60d9f"
format_stamp: "Formatted at 2026-08-12 08:02:12 on dist-test-slave-ncp9"
I20260812 08:02:12.917248 16489 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "667d51c376fb4f57bac5ccd43eb60d9f"
format_stamp: "Formatted at 2026-08-12 08:02:12 on dist-test-slave-ncp9"
I20260812 08:02:12.920439 16489 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:02:12.922525 16513 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:12.923448 16489 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:02:12.923559 16489 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "667d51c376fb4f57bac5ccd43eb60d9f"
format_stamp: "Formatted at 2026-08-12 08:02:12 on dist-test-slave-ncp9"
I20260812 08:02:12.923641 16489 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:12.941586 16489 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:12.942317 16489 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:12.942513 16489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:12.943156 16489 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:12.944248 16489 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:12.944304 16489 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:12.944366 16489 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:12.944394 16489 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:12.954254 16489 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.129:41247
I20260812 08:02:12.954300 16717 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.129:41247 every 8 connection(s)
I20260812 08:02:12.955312 16489 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:02:12.963088  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 16489
I20260812 08:02:12.963228  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:02:12.964566 16719 heartbeater.cc:344] Connected to a master server at 127.5.142.190:33763
I20260812 08:02:12.964828 16719 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:12.965514 16719 heartbeater.cc:507] Master 127.5.142.190:33763 requested a full tablet report, sending...
I20260812 08:02:12.965740  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.130:0
--local_ip_for_outbound_sockets=127.5.142.130
--webserver_interface=127.5.142.130
--webserver_port=0
--tserver_master_addrs=127.5.142.190:33763
--builtin_ntp_servers=127.5.142.148:37867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--enable_leader_failure_detection=false with env {}
I20260812 08:02:12.967113 16375 ts_manager.cc:194] Registered new tserver with Master: 667d51c376fb4f57bac5ccd43eb60d9f (127.5.142.129:41247)
I20260812 08:02:12.968322 16375 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.129:41571
W20260812 08:02:13.072887 16724 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 08:02:13.073156 16724 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:13.073180 16724 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:13.073218 16724 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:13.076509 16724 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:13.076841 16724 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.130
I20260812 08:02:13.081147 16724 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:37867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.5.142.130
--webserver_port=0
--tserver_master_addrs=127.5.142.190:33763
--never_fsync=true
--heap_profile_path=/tmp/kudu.16724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.130

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:13.082341 16724 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:13.083547 16724 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:02:13.086524 16724 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:02:13.090363 16731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:13.090318 16737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:13.090338 16734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:13.090407 16724 server_base.cc:1061] running on GCE node
I20260812 08:02:13.090943 16724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:13.091521 16724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:13.092676 16724 hybrid_clock.cc:648] HybridClock initialized: now 1786521733092655 us; error 37 us; skew 500 ppm
I20260812 08:02:13.094426 16724 webserver.cc:533] Webserver started at http://127.5.142.130:36055/ using document root <none> and password file <none>
I20260812 08:02:13.094993 16724 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:13.095052 16724 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:13.095239 16724 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:13.096794 16724 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "0e86a5d77dea44b0b598901fdc6b6276"
format_stamp: "Formatted at 2026-08-12 08:02:13 on dist-test-slave-ncp9"
I20260812 08:02:13.097321 16724 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "0e86a5d77dea44b0b598901fdc6b6276"
format_stamp: "Formatted at 2026-08-12 08:02:13 on dist-test-slave-ncp9"
I20260812 08:02:13.100566 16724 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:02:13.102718 16744 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:13.103699 16724 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:13.103803 16724 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "0e86a5d77dea44b0b598901fdc6b6276"
format_stamp: "Formatted at 2026-08-12 08:02:13 on dist-test-slave-ncp9"
I20260812 08:02:13.103886 16724 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:13.118719 16724 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:13.119472 16724 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:13.119670 16724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:13.120323 16724 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:13.121491 16724 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:13.121551 16724 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:13.121608 16724 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:13.121632 16724 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:13.130669 16724 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.130:40009
I20260812 08:02:13.130708 16934 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.130:40009 every 8 connection(s)
I20260812 08:02:13.131667 16724 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:02:13.134150  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 16724
I20260812 08:02:13.134261  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:02:13.136631  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.131:0
--local_ip_for_outbound_sockets=127.5.142.131
--webserver_interface=127.5.142.131
--webserver_port=0
--tserver_master_addrs=127.5.142.190:33763
--builtin_ntp_servers=127.5.142.148:37867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--enable_leader_failure_detection=false with env {}
I20260812 08:02:13.140404 16935 heartbeater.cc:344] Connected to a master server at 127.5.142.190:33763
I20260812 08:02:13.140683 16935 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:13.141319 16935 heartbeater.cc:507] Master 127.5.142.190:33763 requested a full tablet report, sending...
I20260812 08:02:13.142462 16375 ts_manager.cc:194] Registered new tserver with Master: 0e86a5d77dea44b0b598901fdc6b6276 (127.5.142.130:40009)
I20260812 08:02:13.143057 16375 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.130:46611
W20260812 08:02:13.246994 16940 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 08:02:13.247222 16940 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:13.247244 16940 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:13.247292 16940 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:13.250725 16940 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:13.250835 16940 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.131
I20260812 08:02:13.254693 16940 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:37867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.5.142.131
--webserver_port=0
--tserver_master_addrs=127.5.142.190:33763
--never_fsync=true
--heap_profile_path=/tmp/kudu.16940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.131

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:13.255755 16940 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:13.256866 16940 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 08:02:13.259708 16940 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 08:02:13.263235 16948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:13.263212 16952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:13.263438 16940 server_base.cc:1061] running on GCE node
W20260812 08:02:13.263226 16949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:13.263859 16940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:13.264381 16940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:13.265565 16940 hybrid_clock.cc:648] HybridClock initialized: now 1786521733265548 us; error 33 us; skew 500 ppm
I20260812 08:02:13.267324 16940 webserver.cc:533] Webserver started at http://127.5.142.131:36649/ using document root <none> and password file <none>
I20260812 08:02:13.267887 16940 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:13.267942 16940 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:13.268141 16940 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:13.269747 16940 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "3defb91e40be4fa59c52b048755d3eba"
format_stamp: "Formatted at 2026-08-12 08:02:13 on dist-test-slave-ncp9"
I20260812 08:02:13.270210 16940 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "3defb91e40be4fa59c52b048755d3eba"
format_stamp: "Formatted at 2026-08-12 08:02:13 on dist-test-slave-ncp9"
I20260812 08:02:13.273506 16940 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:02:13.275625 16969 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:13.276626 16940 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:13.276721 16940 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "3defb91e40be4fa59c52b048755d3eba"
format_stamp: "Formatted at 2026-08-12 08:02:13 on dist-test-slave-ncp9"
I20260812 08:02:13.276795 16940 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:13.291724 16940 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:13.292349 16940 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:13.292490 16940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:13.293044 16940 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:13.294052 16940 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:13.294100 16940 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:13.294147 16940 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:13.294167 16940 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:13.303051 16940 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.131:38153
I20260812 08:02:13.303099 17161 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.131:38153 every 8 connection(s)
I20260812 08:02:13.304015 16940 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:02:13.306187  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 16940
I20260812 08:02:13.306298  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:02:13.311923 17162 heartbeater.cc:344] Connected to a master server at 127.5.142.190:33763
I20260812 08:02:13.312158 17162 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:13.312640 17162 heartbeater.cc:507] Master 127.5.142.190:33763 requested a full tablet report, sending...
I20260812 08:02:13.313624 16375 ts_manager.cc:194] Registered new tserver with Master: 3defb91e40be4fa59c52b048755d3eba (127.5.142.131:38153)
I20260812 08:02:13.314133 16375 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.131:48303
I20260812 08:02:13.320124  5690 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:02:13.328521 16375 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
W20260812 08:02:13.331056 16375 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 08:02:13.347849 16819 tablet_service.cc:1511] Processing CreateTablet for tablet 5263547ef57d40b6b1ac501be3c6f4cc (DEFAULT_TABLE table=TestTable [id=7ef14fabba8248e0a89d97730bf982a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:13.347906 16599 tablet_service.cc:1511] Processing CreateTablet for tablet 5263547ef57d40b6b1ac501be3c6f4cc (DEFAULT_TABLE table=TestTable [id=7ef14fabba8248e0a89d97730bf982a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:13.348825 16599 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5263547ef57d40b6b1ac501be3c6f4cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:13.348834 16819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5263547ef57d40b6b1ac501be3c6f4cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:13.350811 17046 tablet_service.cc:1511] Processing CreateTablet for tablet 5263547ef57d40b6b1ac501be3c6f4cc (DEFAULT_TABLE table=TestTable [id=7ef14fabba8248e0a89d97730bf982a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:13.351851 17046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5263547ef57d40b6b1ac501be3c6f4cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:13.353984 17186 tablet_bootstrap.cc:492] T 5263547ef57d40b6b1ac501be3c6f4cc P 0e86a5d77dea44b0b598901fdc6b6276: Bootstrap starting.
I20260812 08:02:13.355218 17188 tablet_bootstrap.cc:492] T 5263547ef57d40b6b1ac501be3c6f4cc P 667d51c376fb4f57bac5ccd43eb60d9f: Bootstrap starting.
I20260812 08:02:13.357039 17191 tablet_bootstrap.cc:492] T 5263547ef57d40b6b1ac501be3c6f4cc P 3defb91e40be4fa59c52b048755d3eba: Bootstrap starting.
I20260812 08:02:13.357255 17186 tablet_bootstrap.cc:654] T 5263547ef57d40b6b1ac501be3c6f4cc P 0e86a5d77dea44b0b598901fdc6b6276: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:13.357255 17188 tablet_bootstrap.cc:654] T 5263547ef57d40b6b1ac501be3c6f4cc P 667d51c376fb4f57bac5ccd43eb60d9f: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:13.358108 17188 log.cc:826] T 5263547ef57d40b6b1ac501be3c6f4cc P 667d51c376fb4f57bac5ccd43eb60d9f: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:13.358108 17186 log.cc:826] T 5263547ef57d40b6b1ac501be3c6f4cc P 0e86a5d77dea44b0b598901fdc6b6276: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:13.359055 17191 tablet_bootstrap.cc:654] T 5263547ef57d40b6b1ac501be3c6f4cc P 3defb91e40be4fa59c52b048755d3eba: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:13.359668 17191 log.cc:826] T 5263547ef57d40b6b1ac501be3c6f4cc P 3defb91e40be4fa59c52b048755d3eba: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:13.359808 17188 tablet_bootstrap.cc:492] T 5263547ef57d40b6b1ac501be3c6f4cc P 667d51c376fb4f57bac5ccd43eb60d9f: No bootstrap required, opened a new log
I20260812 08:02:13.359808 17186 tablet_bootstrap.cc:492] T 5263547ef57d40b6b1ac501be3c6f4cc P 0e86a5d77dea44b0b598901fdc6b6276: No bootstrap required, opened a new log
I20260812 08:02:13.359995 17186 ts_tablet_manager.cc:1403] T 5263547ef57d40b6b1ac501be3c6f4cc P 0e86a5d77dea44b0b598901fdc6b6276: Time spent bootstrapping tablet: real 0.006s	user 0.004s	sys 0.000s
I20260812 08:02:13.360014 17188 ts_tablet_manager.cc:1403] T 5263547ef57d40b6b1ac501be3c6f4cc P 667d51c376fb4f57bac5ccd43eb60d9f: Time spent bootstrapping tablet: real 0.005s	user 0.003s	sys 0.000s
I20260812 08:02:13.361150 17191 tablet_bootstrap.cc:492] T 5263547ef57d40b6b1ac501be3c6f4cc P 3defb91e40be4fa59c52b048755d3eba: No bootstrap required, opened a new log
I20260812 08:02:13.361327 17191 ts_tablet_manager.cc:1403] T 5263547ef57d40b6b1ac501be3c6f4cc P 3defb91e40be4fa59c52b048755d3eba: Time spent bootstrapping tablet: real 0.004s	user 0.004s	sys 0.000s
I20260812 08:02:13.363039 17188 raft_consensus.cc:359] T 5263547ef57d40b6b1ac501be3c6f4cc P 667d51c376fb4f57bac5ccd43eb60d9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e86a5d77dea44b0b598901fdc6b6276" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 40009 } } peers { permanent_uuid: "667d51c376fb4f57bac5ccd43eb60d9f" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41247 } } peers { permanent_uuid: "3defb91e40be4fa59c52b048755d3eba" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 38153 } }
I20260812 08:02:13.363039 17186 raft_consensus.cc:359] T 5263547ef57d40b6b1ac501be3c6f4cc P 0e86a5d77dea44b0b598901fdc6b6276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e86a5d77dea44b0b598901fdc6b6276" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 40009 } } peers { permanent_uuid: "667d51c376fb4f57bac5ccd43eb60d9f" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41247 } } peers { permanent_uuid: "3defb91e40be4fa59c52b048755d3eba" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 38153 } }
I20260812 08:02:13.363247 17186 raft_consensus.cc:740] T 5263547ef57d40b6b1ac501be3c6f4cc P 0e86a5d77dea44b0b598901fdc6b6276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e86a5d77dea44b0b598901fdc6b6276, State: Initialized, Role: FOLLOWER
I20260812 08:02:13.363258 17188 raft_consensus.cc:740] T 5263547ef57d40b6b1ac501be3c6f4cc P 667d51c376fb4f57bac5ccd43eb60d9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 667d51c376fb4f57bac5ccd43eb60d9f, State: Initialized, Role: FOLLOWER
I20260812 08:02:13.363569 17188 consensus_queue.cc:260] T 5263547ef57d40b6b1ac501be3c6f4cc P 667d51c376fb4f57bac5ccd43eb60d9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e86a5d77dea44b0b598901fdc6b6276" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 40009 } } peers { permanent_uuid: "667d51c376fb4f57bac5ccd43eb60d9f" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41247 } } peers { permanent_uuid: "3defb91e40be4fa59c52b048755d3eba" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 38153 } }
I20260812 08:02:13.363569 17186 consensus_queue.cc:260] T 5263547ef57d40b6b1ac501be3c6f4cc P 0e86a5d77dea44b0b598901fdc6b6276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e86a5d77dea44b0b598901fdc6b6276" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 40009 } } peers { permanent_uuid: "667d51c376fb4f57bac5ccd43eb60d9f" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41247 } } peers { permanent_uuid: "3defb91e40be4fa59c52b048755d3eba" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 38153 } }
I20260812 08:02:13.363731 17191 raft_consensus.cc:359] T 5263547ef57d40b6b1ac501be3c6f4cc P 3defb91e40be4fa59c52b048755d3eba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e86a5d77dea44b0b598901fdc6b6276" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 40009 } } peers { permanent_uuid: "667d51c376fb4f57bac5ccd43eb60d9f" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41247 } } peers { permanent_uuid: "3defb91e40be4fa59c52b048755d3eba" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 38153 } }
I20260812 08:02:13.363909 17191 raft_consensus.cc:740] T 5263547ef57d40b6b1ac501be3c6f4cc P 3defb91e40be4fa59c52b048755d3eba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3defb91e40be4fa59c52b048755d3eba, State: Initialized, Role: FOLLOWER
I20260812 08:02:13.364207 17191 consensus_queue.cc:260] T 5263547ef57d40b6b1ac501be3c6f4cc P 3defb91e40be4fa59c52b048755d3eba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e86a5d77dea44b0b598901fdc6b6276" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 40009 } } peers { permanent_uuid: "667d51c376fb4f57bac5ccd43eb60d9f" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41247 } } peers { permanent_uuid: "3defb91e40be4fa59c52b048755d3eba" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 38153 } }
I20260812 08:02:13.364355 16935 heartbeater.cc:499] Master 127.5.142.190:33763 was elected leader, sending a full tablet report...
I20260812 08:02:13.364421 17188 ts_tablet_manager.cc:1434] T 5263547ef57d40b6b1ac501be3c6f4cc P 667d51c376fb4f57bac5ccd43eb60d9f: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260812 08:02:13.364741 16719 heartbeater.cc:499] Master 127.5.142.190:33763 was elected leader, sending a full tablet report...
I20260812 08:02:13.365063 17191 ts_tablet_manager.cc:1434] T 5263547ef57d40b6b1ac501be3c6f4cc P 3defb91e40be4fa59c52b048755d3eba: Time spent starting tablet: real 0.004s	user 0.003s	sys 0.000s
I20260812 08:02:13.365391 17162 heartbeater.cc:499] Master 127.5.142.190:33763 was elected leader, sending a full tablet report...
I20260812 08:02:13.365815 17186 ts_tablet_manager.cc:1434] T 5263547ef57d40b6b1ac501be3c6f4cc P 0e86a5d77dea44b0b598901fdc6b6276: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260812 08:02:13.380837  5690 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:02:13.382455  5690 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 667d51c376fb4f57bac5ccd43eb60d9f to finish bootstrapping
W20260812 08:02:13.382550 16936 tablet.cc:2406] T 5263547ef57d40b6b1ac501be3c6f4cc P 0e86a5d77dea44b0b598901fdc6b6276: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:02:13.390018  5690 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 0e86a5d77dea44b0b598901fdc6b6276 to finish bootstrapping
I20260812 08:02:13.397814  5690 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 3defb91e40be4fa59c52b048755d3eba to finish bootstrapping
I20260812 08:02:13.408517 16855 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5263547ef57d40b6b1ac501be3c6f4cc"
dest_uuid: "0e86a5d77dea44b0b598901fdc6b6276"
 from {username='slave'} at 127.0.0.1:35096
I20260812 08:02:13.408741 16855 raft_consensus.cc:493] T 5263547ef57d40b6b1ac501be3c6f4cc P 0e86a5d77dea44b0b598901fdc6b6276 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 08:02:13.408854 16855 raft_consensus.cc:3060] T 5263547ef57d40b6b1ac501be3c6f4cc P 0e86a5d77dea44b0b598901fdc6b6276 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:13.409968 16855 raft_consensus.cc:515] T 5263547ef57d40b6b1ac501be3c6f4cc P 0e86a5d77dea44b0b598901fdc6b6276 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e86a5d77dea44b0b598901fdc6b6276" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 40009 } } peers { permanent_uuid: "667d51c376fb4f57bac5ccd43eb60d9f" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41247 } } peers { permanent_uuid: "3defb91e40be4fa59c52b048755d3eba" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 38153 } }
I20260812 08:02:13.411155 16855 leader_election.cc:290] T 5263547ef57d40b6b1ac501be3c6f4cc P 0e86a5d77dea44b0b598901fdc6b6276 [CANDIDATE]: Term 1 election: Requested vote from peers 667d51c376fb4f57bac5ccd43eb60d9f (127.5.142.129:41247), 3defb91e40be4fa59c52b048755d3eba (127.5.142.131:38153)
I20260812 08:02:13.415439 16633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5263547ef57d40b6b1ac501be3c6f4cc" candidate_uuid: "0e86a5d77dea44b0b598901fdc6b6276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "667d51c376fb4f57bac5ccd43eb60d9f"
I20260812 08:02:13.415706 16633 raft_consensus.cc:3060] T 5263547ef57d40b6b1ac501be3c6f4cc P 667d51c376fb4f57bac5ccd43eb60d9f [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:13.416128 17076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5263547ef57d40b6b1ac501be3c6f4cc" candidate_uuid: "0e86a5d77dea44b0b598901fdc6b6276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3defb91e40be4fa59c52b048755d3eba"
I20260812 08:02:13.416312 17076 raft_consensus.cc:3060] T 5263547ef57d40b6b1ac501be3c6f4cc P 3defb91e40be4fa59c52b048755d3eba [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:13.417057 16633 raft_consensus.cc:2468] T 5263547ef57d40b6b1ac501be3c6f4cc P 667d51c376fb4f57bac5ccd43eb60d9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e86a5d77dea44b0b598901fdc6b6276 in term 1.
I20260812 08:02:13.417349 17076 raft_consensus.cc:2468] T 5263547ef57d40b6b1ac501be3c6f4cc P 3defb91e40be4fa59c52b048755d3eba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e86a5d77dea44b0b598901fdc6b6276 in term 1.
I20260812 08:02:13.417725 16750 leader_election.cc:304] T 5263547ef57d40b6b1ac501be3c6f4cc P 0e86a5d77dea44b0b598901fdc6b6276 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e86a5d77dea44b0b598901fdc6b6276, 667d51c376fb4f57bac5ccd43eb60d9f; no voters: 
I20260812 08:02:13.418324 17199 raft_consensus.cc:2804] T 5263547ef57d40b6b1ac501be3c6f4cc P 0e86a5d77dea44b0b598901fdc6b6276 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:02:13.418552 17199 raft_consensus.cc:697] T 5263547ef57d40b6b1ac501be3c6f4cc P 0e86a5d77dea44b0b598901fdc6b6276 [term 1 LEADER]: Becoming Leader. State: Replica: 0e86a5d77dea44b0b598901fdc6b6276, State: Running, Role: LEADER
I20260812 08:02:13.418892 17199 consensus_queue.cc:237] T 5263547ef57d40b6b1ac501be3c6f4cc P 0e86a5d77dea44b0b598901fdc6b6276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e86a5d77dea44b0b598901fdc6b6276" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 40009 } } peers { permanent_uuid: "667d51c376fb4f57bac5ccd43eb60d9f" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41247 } } peers { permanent_uuid: "3defb91e40be4fa59c52b048755d3eba" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 38153 } }
I20260812 08:02:13.424083 16371 catalog_manager.cc:5719] T 5263547ef57d40b6b1ac501be3c6f4cc P 0e86a5d77dea44b0b598901fdc6b6276 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e86a5d77dea44b0b598901fdc6b6276 (127.5.142.130). New cstate: current_term: 1 leader_uuid: "0e86a5d77dea44b0b598901fdc6b6276" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e86a5d77dea44b0b598901fdc6b6276" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 40009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "667d51c376fb4f57bac5ccd43eb60d9f" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3defb91e40be4fa59c52b048755d3eba" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 38153 } health_report { overall_health: UNKNOWN } } }
W20260812 08:02:13.456535 16721 tablet.cc:2406] T 5263547ef57d40b6b1ac501be3c6f4cc P 667d51c376fb4f57bac5ccd43eb60d9f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:02:13.511890 17199 consensus_queue.cc:1048] T 5263547ef57d40b6b1ac501be3c6f4cc P 0e86a5d77dea44b0b598901fdc6b6276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "667d51c376fb4f57bac5ccd43eb60d9f" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 41247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:02:13.517627 17199 consensus_queue.cc:1048] T 5263547ef57d40b6b1ac501be3c6f4cc P 0e86a5d77dea44b0b598901fdc6b6276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3defb91e40be4fa59c52b048755d3eba" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 38153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:02:13.529496  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 16489
I20260812 08:02:13.546340  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 16724
I20260812 08:02:14.068090  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 16940
I20260812 08:02:14.085173  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 16321
2026-08-12T08:02:14Z chronyd exiting
[       OK ] RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled (1561 ms)
[----------] 6 tests from RaftConsensusITest (82835 ms total)

[----------] 1 test from EncryptionEnabled/RaftConsensusParamEncryptionITest
[ RUN      ] EncryptionEnabled/RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC/0
I20260812 08:02:14.112869  5690 ts_itest-base.cc:115] Starting cluster with:
I20260812 08:02:14.112975  5690 ts_itest-base.cc:116] --------------
I20260812 08:02:14.112991  5690 ts_itest-base.cc:117] 3 tablet servers
I20260812 08:02:14.113004  5690 ts_itest-base.cc:118] 3 replicas per TS
I20260812 08:02:14.113018  5690 ts_itest-base.cc:119] --------------
2026-08-12T08:02:14Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:02:14Z Disabled control of system clock
I20260812 08:02:14.125139  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.142.190:37039
--webserver_interface=127.5.142.190
--webserver_port=0
--builtin_ntp_servers=127.5.142.148:46699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.142.190:37039 with env {}
W20260812 08:02:14.232141 17237 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:14.232362 17237 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:14.232416 17237 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:14.235674 17237 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:02:14.235728 17237 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:14.235754 17237 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:02:14.235776 17237 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:02:14.239748 17237 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:46699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.142.190:37039
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.142.190:37039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.5.142.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.17237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:14.240840 17237 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:14.241972 17237 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:14.247469 17248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:14.247519 17244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:14.247560 17245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:14.247779 17237 server_base.cc:1061] running on GCE node
I20260812 08:02:14.248253 17237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:14.249186 17237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:14.250334 17237 hybrid_clock.cc:648] HybridClock initialized: now 1786521734250320 us; error 31 us; skew 500 ppm
I20260812 08:02:14.251964 17237 webserver.cc:533] Webserver started at http://127.5.142.190:38511/ using document root <none> and password file <none>
I20260812 08:02:14.252496 17237 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:14.252554 17237 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:14.252743 17237 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:14.254251 17237 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "050240c1fbd242e5818053cbafce4da7"
format_stamp: "Formatted at 2026-08-12 08:02:14 on dist-test-slave-ncp9"
I20260812 08:02:14.254694 17237 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "050240c1fbd242e5818053cbafce4da7"
format_stamp: "Formatted at 2026-08-12 08:02:14 on dist-test-slave-ncp9"
I20260812 08:02:14.257825 17237 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:02:14.259887 17258 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:14.260815 17237 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:14.260942 17237 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
uuid: "050240c1fbd242e5818053cbafce4da7"
format_stamp: "Formatted at 2026-08-12 08:02:14 on dist-test-slave-ncp9"
I20260812 08:02:14.261030 17237 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:14.273437 17237 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:14.274127 17237 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:14.274296 17237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:14.281770 17351 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.190:37039 every 8 connection(s)
I20260812 08:02:14.281772 17237 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.190:37039
I20260812 08:02:14.282863 17237 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 08:02:14.285784 17353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:14.290824 17353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 050240c1fbd242e5818053cbafce4da7: Bootstrap starting.
I20260812 08:02:14.291563  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 17237
I20260812 08:02:14.291708  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 08:02:14.293504 17353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 050240c1fbd242e5818053cbafce4da7: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:14.294203 17353 log.cc:826] T 00000000000000000000000000000000 P 050240c1fbd242e5818053cbafce4da7: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:14.295794 17353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 050240c1fbd242e5818053cbafce4da7: No bootstrap required, opened a new log
I20260812 08:02:14.298636 17353 raft_consensus.cc:359] T 00000000000000000000000000000000 P 050240c1fbd242e5818053cbafce4da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050240c1fbd242e5818053cbafce4da7" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 37039 } }
I20260812 08:02:14.298882 17353 raft_consensus.cc:385] T 00000000000000000000000000000000 P 050240c1fbd242e5818053cbafce4da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:14.298959 17353 raft_consensus.cc:740] T 00000000000000000000000000000000 P 050240c1fbd242e5818053cbafce4da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 050240c1fbd242e5818053cbafce4da7, State: Initialized, Role: FOLLOWER
I20260812 08:02:14.299397 17353 consensus_queue.cc:260] T 00000000000000000000000000000000 P 050240c1fbd242e5818053cbafce4da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050240c1fbd242e5818053cbafce4da7" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 37039 } }
I20260812 08:02:14.299526 17353 raft_consensus.cc:399] T 00000000000000000000000000000000 P 050240c1fbd242e5818053cbafce4da7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:14.299602 17353 raft_consensus.cc:493] T 00000000000000000000000000000000 P 050240c1fbd242e5818053cbafce4da7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:14.299717 17353 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 050240c1fbd242e5818053cbafce4da7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:14.300700 17353 raft_consensus.cc:515] T 00000000000000000000000000000000 P 050240c1fbd242e5818053cbafce4da7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050240c1fbd242e5818053cbafce4da7" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 37039 } }
I20260812 08:02:14.301083 17353 leader_election.cc:304] T 00000000000000000000000000000000 P 050240c1fbd242e5818053cbafce4da7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 050240c1fbd242e5818053cbafce4da7; no voters: 
I20260812 08:02:14.301369 17353 leader_election.cc:290] T 00000000000000000000000000000000 P 050240c1fbd242e5818053cbafce4da7 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:02:14.301476 17366 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 050240c1fbd242e5818053cbafce4da7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:02:14.301671 17366 raft_consensus.cc:697] T 00000000000000000000000000000000 P 050240c1fbd242e5818053cbafce4da7 [term 1 LEADER]: Becoming Leader. State: Replica: 050240c1fbd242e5818053cbafce4da7, State: Running, Role: LEADER
I20260812 08:02:14.301951 17366 consensus_queue.cc:237] T 00000000000000000000000000000000 P 050240c1fbd242e5818053cbafce4da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050240c1fbd242e5818053cbafce4da7" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 37039 } }
I20260812 08:02:14.302647 17353 sys_catalog.cc:565] T 00000000000000000000000000000000 P 050240c1fbd242e5818053cbafce4da7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:02:14.303601 17367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 050240c1fbd242e5818053cbafce4da7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "050240c1fbd242e5818053cbafce4da7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050240c1fbd242e5818053cbafce4da7" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 37039 } } }
I20260812 08:02:14.303568 17368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 050240c1fbd242e5818053cbafce4da7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 050240c1fbd242e5818053cbafce4da7. Latest consensus state: current_term: 1 leader_uuid: "050240c1fbd242e5818053cbafce4da7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050240c1fbd242e5818053cbafce4da7" member_type: VOTER last_known_addr { host: "127.5.142.190" port: 37039 } } }
I20260812 08:02:14.303740 17367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 050240c1fbd242e5818053cbafce4da7 [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:14.303741 17368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 050240c1fbd242e5818053cbafce4da7 [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:14.304734 17380 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:02:14.307665 17380 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:02:14.313220 17380 catalog_manager.cc:1383] Generated new cluster ID: b918242826c4415986d8ab21c588361b
I20260812 08:02:14.313288 17380 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:02:14.343248 17380 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:02:14.344261 17380 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:02:14.357265 17380 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 050240c1fbd242e5818053cbafce4da7: Generated new TSK 0
I20260812 08:02:14.357882 17380 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:02:14.368413  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.129:0
--local_ip_for_outbound_sockets=127.5.142.129
--webserver_interface=127.5.142.129
--webserver_port=0
--tserver_master_addrs=127.5.142.190:37039
--builtin_ntp_servers=127.5.142.148:46699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--evict_failed_followers=false
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20260812 08:02:14.485669 17398 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:02:14.485903 17398 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:14.485929 17398 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:14.485970 17398 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:14.489228 17398 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:02:14.489279 17398 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:02:14.489315 17398 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:14.489377 17398 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:02:14.489403 17398 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.129
I20260812 08:02:14.493187 17398 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:46699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--evict_failed_followers=false
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.5.142.129
--webserver_port=0
--tserver_master_addrs=127.5.142.190:37039
--never_fsync=true
--heap_profile_path=/tmp/kudu.17398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.129

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:14.494202 17398 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:14.495327 17398 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:14.501823 17407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:14.501818 17405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:14.501854 17409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:14.502111 17398 server_base.cc:1061] running on GCE node
I20260812 08:02:14.502539 17398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:14.503098 17398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:14.504256 17398 hybrid_clock.cc:648] HybridClock initialized: now 1786521734504243 us; error 25 us; skew 500 ppm
I20260812 08:02:14.506064 17398 webserver.cc:533] Webserver started at http://127.5.142.129:45221/ using document root <none> and password file <none>
I20260812 08:02:14.506650 17398 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:14.506707 17398 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:14.506910 17398 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:14.508512 17398 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "f92587a8c94d4ac49cca93637e537626"
format_stamp: "Formatted at 2026-08-12 08:02:14 on dist-test-slave-ncp9"
I20260812 08:02:14.509006 17398 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "f92587a8c94d4ac49cca93637e537626"
format_stamp: "Formatted at 2026-08-12 08:02:14 on dist-test-slave-ncp9"
I20260812 08:02:14.512283 17398 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:02:14.514580 17417 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:14.515559 17398 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.001s
I20260812 08:02:14.515671 17398 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "f92587a8c94d4ac49cca93637e537626"
format_stamp: "Formatted at 2026-08-12 08:02:14 on dist-test-slave-ncp9"
I20260812 08:02:14.515758 17398 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:14.544656 17398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:14.545439 17398 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:14.545619 17398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:14.546226 17398 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:14.547260 17398 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:14.547310 17398 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:14.547365 17398 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:14.547391 17398 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:14.556149 17398 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.129:39477
I20260812 08:02:14.556174 17591 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.129:39477 every 8 connection(s)
I20260812 08:02:14.557116 17398 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 08:02:14.558212  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 17398
I20260812 08:02:14.558326  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 08:02:14.560811  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.130:0
--local_ip_for_outbound_sockets=127.5.142.130
--webserver_interface=127.5.142.130
--webserver_port=0
--tserver_master_addrs=127.5.142.190:37039
--builtin_ntp_servers=127.5.142.148:46699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--evict_failed_followers=false
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20260812 08:02:14.566256 17593 heartbeater.cc:344] Connected to a master server at 127.5.142.190:37039
I20260812 08:02:14.566511 17593 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:14.567116 17593 heartbeater.cc:507] Master 127.5.142.190:37039 requested a full tablet report, sending...
I20260812 08:02:14.568681 17284 ts_manager.cc:194] Registered new tserver with Master: f92587a8c94d4ac49cca93637e537626 (127.5.142.129:39477)
I20260812 08:02:14.569834 17284 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.129:36543
W20260812 08:02:14.668368 17600 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:02:14.668607 17600 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:14.668655 17600 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:14.668718 17600 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:14.671940 17600 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:02:14.671994 17600 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:02:14.672041 17600 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:14.672125 17600 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:02:14.672159 17600 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.130
I20260812 08:02:14.675923 17600 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:46699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--evict_failed_followers=false
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.5.142.130
--webserver_port=0
--tserver_master_addrs=127.5.142.190:37039
--never_fsync=true
--heap_profile_path=/tmp/kudu.17600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.130

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:14.676996 17600 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:14.678098 17600 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:14.684341 17605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:14.684376 17606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:14.684406 17608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:14.684739 17600 server_base.cc:1061] running on GCE node
I20260812 08:02:14.685266 17600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:14.685864 17600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:14.686990 17600 hybrid_clock.cc:648] HybridClock initialized: now 1786521734686932 us; error 74 us; skew 500 ppm
I20260812 08:02:14.688931 17600 webserver.cc:533] Webserver started at http://127.5.142.130:43267/ using document root <none> and password file <none>
I20260812 08:02:14.689549 17600 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:14.689611 17600 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:14.689803 17600 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:14.691378 17600 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "c3f2977be270425889284dd0f2207502"
format_stamp: "Formatted at 2026-08-12 08:02:14 on dist-test-slave-ncp9"
I20260812 08:02:14.691851 17600 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "c3f2977be270425889284dd0f2207502"
format_stamp: "Formatted at 2026-08-12 08:02:14 on dist-test-slave-ncp9"
I20260812 08:02:14.695410 17600 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:02:14.697878 17618 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:14.698931 17600 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:02:14.699050 17600 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "c3f2977be270425889284dd0f2207502"
format_stamp: "Formatted at 2026-08-12 08:02:14 on dist-test-slave-ncp9"
I20260812 08:02:14.699146 17600 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:14.717108 17600 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:14.717751 17600 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:14.717896 17600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:14.718422 17600 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:14.719413 17600 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:14.719461 17600 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:14.719501 17600 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:14.719532 17600 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:14.728473 17600 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.130:33501
I20260812 08:02:14.728497 17812 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.130:33501 every 8 connection(s)
I20260812 08:02:14.729460 17600 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 08:02:14.729947  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 17600
I20260812 08:02:14.730086  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 08:02:14.732606  5690 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskapknlq/build/debug/bin/kudu
/tmp/dist-test-taskapknlq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.142.131:0
--local_ip_for_outbound_sockets=127.5.142.131
--webserver_interface=127.5.142.131
--webserver_port=0
--tserver_master_addrs=127.5.142.190:37039
--builtin_ntp_servers=127.5.142.148:46699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--evict_failed_followers=false
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20260812 08:02:14.738194 17813 heartbeater.cc:344] Connected to a master server at 127.5.142.190:37039
I20260812 08:02:14.738432 17813 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:14.738943 17813 heartbeater.cc:507] Master 127.5.142.190:37039 requested a full tablet report, sending...
I20260812 08:02:14.739976 17284 ts_manager.cc:194] Registered new tserver with Master: c3f2977be270425889284dd0f2207502 (127.5.142.130:33501)
I20260812 08:02:14.740522 17284 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.130:44353
W20260812 08:02:14.842447 17816 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 08:02:14.842742 17816 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:02:14.842782 17816 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:02:14.842846 17816 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:02:14.846264 17816 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 08:02:14.846323 17816 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 08:02:14.846372 17816 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:02:14.846457 17816 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 08:02:14.846493 17816 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.142.131
I20260812 08:02:14.850550 17816 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.142.148:46699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--evict_failed_followers=false
--fs_data_dirs=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.142.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.5.142.131
--webserver_port=0
--tserver_master_addrs=127.5.142.190:37039
--never_fsync=true
--heap_profile_path=/tmp/kudu.17816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.142.131

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:02:14.851661 17816 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:14.852869 17816 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:14.859179 17823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:14.859297 17830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:14.859297 17824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:14.859858 17816 server_base.cc:1061] running on GCE node
I20260812 08:02:14.860208 17816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:02:14.860803 17816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:02:14.861980 17816 hybrid_clock.cc:648] HybridClock initialized: now 1786521734861962 us; error 28 us; skew 500 ppm
I20260812 08:02:14.863802 17816 webserver.cc:533] Webserver started at http://127.5.142.131:38599/ using document root <none> and password file <none>
I20260812 08:02:14.864348 17816 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:14.864408 17816 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:14.864598 17816 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:14.866191 17816 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "054c58374a914c0eabd9da4986000865"
format_stamp: "Formatted at 2026-08-12 08:02:14 on dist-test-slave-ncp9"
I20260812 08:02:14.866659 17816 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "054c58374a914c0eabd9da4986000865"
format_stamp: "Formatted at 2026-08-12 08:02:14 on dist-test-slave-ncp9"
I20260812 08:02:14.870219 17816 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:02:14.872462 17843 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:14.873554 17816 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:02:14.873668 17816 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "054c58374a914c0eabd9da4986000865"
format_stamp: "Formatted at 2026-08-12 08:02:14 on dist-test-slave-ncp9"
I20260812 08:02:14.873759 17816 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:14.892436 17816 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:14.893198 17816 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:14.893375 17816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:14.894014 17816 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:14.895097 17816 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:14.895149 17816 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:14.895205 17816 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:14.895236 17816 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:14.904173 17816 rpc_server.cc:307] RPC server started. Bound to: 127.5.142.131:45873
I20260812 08:02:14.904222 18046 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.142.131:45873 every 8 connection(s)
I20260812 08:02:14.905200 17816 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 08:02:14.911295  5690 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskapknlq/build/debug/bin/kudu as pid 17816
I20260812 08:02:14.911399  5690 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 08:02:14.913687 18049 heartbeater.cc:344] Connected to a master server at 127.5.142.190:37039
I20260812 08:02:14.913918 18049 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:14.914400 18049 heartbeater.cc:507] Master 127.5.142.190:37039 requested a full tablet report, sending...
I20260812 08:02:14.915382 17284 ts_manager.cc:194] Registered new tserver with Master: 054c58374a914c0eabd9da4986000865 (127.5.142.131:45873)
I20260812 08:02:14.915915 17284 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.142.131:38535
I20260812 08:02:14.925981  5690 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:02:14.934796 17284 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37918:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
W20260812 08:02:14.937345 17284 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 08:02:14.953330 17490 tablet_service.cc:1511] Processing CreateTablet for tablet 0cf52873c58c4be6b624dec80436ef4c (DEFAULT_TABLE table=TestTable [id=21894f80755647b1aa91914be089f599]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:14.953337 17704 tablet_service.cc:1511] Processing CreateTablet for tablet 0cf52873c58c4be6b624dec80436ef4c (DEFAULT_TABLE table=TestTable [id=21894f80755647b1aa91914be089f599]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:14.953338 17929 tablet_service.cc:1511] Processing CreateTablet for tablet 0cf52873c58c4be6b624dec80436ef4c (DEFAULT_TABLE table=TestTable [id=21894f80755647b1aa91914be089f599]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:02:14.954241 17490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cf52873c58c4be6b624dec80436ef4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:14.954252 17929 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cf52873c58c4be6b624dec80436ef4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:14.954252 17704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cf52873c58c4be6b624dec80436ef4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:14.959092 18075 tablet_bootstrap.cc:492] T 0cf52873c58c4be6b624dec80436ef4c P f92587a8c94d4ac49cca93637e537626: Bootstrap starting.
I20260812 08:02:14.960291 18077 tablet_bootstrap.cc:492] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865: Bootstrap starting.
I20260812 08:02:14.960292 18076 tablet_bootstrap.cc:492] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502: Bootstrap starting.
I20260812 08:02:14.961658 18075 tablet_bootstrap.cc:654] T 0cf52873c58c4be6b624dec80436ef4c P f92587a8c94d4ac49cca93637e537626: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:14.962240 18077 tablet_bootstrap.cc:654] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:14.962360 18076 tablet_bootstrap.cc:654] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:14.962519 18075 log.cc:826] T 0cf52873c58c4be6b624dec80436ef4c P f92587a8c94d4ac49cca93637e537626: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:14.962932 18077 log.cc:826] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:14.963135 18076 log.cc:826] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:14.964197 18075 tablet_bootstrap.cc:492] T 0cf52873c58c4be6b624dec80436ef4c P f92587a8c94d4ac49cca93637e537626: No bootstrap required, opened a new log
I20260812 08:02:14.964270 18077 tablet_bootstrap.cc:492] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865: No bootstrap required, opened a new log
I20260812 08:02:14.964375 18075 ts_tablet_manager.cc:1403] T 0cf52873c58c4be6b624dec80436ef4c P f92587a8c94d4ac49cca93637e537626: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:02:14.964435 18077 ts_tablet_manager.cc:1403] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865: Time spent bootstrapping tablet: real 0.004s	user 0.004s	sys 0.000s
I20260812 08:02:14.965766 18076 tablet_bootstrap.cc:492] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502: No bootstrap required, opened a new log
I20260812 08:02:14.965952 18076 ts_tablet_manager.cc:1403] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502: Time spent bootstrapping tablet: real 0.006s	user 0.004s	sys 0.000s
I20260812 08:02:14.967068 18075 raft_consensus.cc:359] T 0cf52873c58c4be6b624dec80436ef4c P f92587a8c94d4ac49cca93637e537626 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 } } peers { permanent_uuid: "c3f2977be270425889284dd0f2207502" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33501 } } peers { permanent_uuid: "054c58374a914c0eabd9da4986000865" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 45873 } }
I20260812 08:02:14.967294 18075 raft_consensus.cc:385] T 0cf52873c58c4be6b624dec80436ef4c P f92587a8c94d4ac49cca93637e537626 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:14.967350 18075 raft_consensus.cc:740] T 0cf52873c58c4be6b624dec80436ef4c P f92587a8c94d4ac49cca93637e537626 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f92587a8c94d4ac49cca93637e537626, State: Initialized, Role: FOLLOWER
I20260812 08:02:14.967397 18077 raft_consensus.cc:359] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 } } peers { permanent_uuid: "c3f2977be270425889284dd0f2207502" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33501 } } peers { permanent_uuid: "054c58374a914c0eabd9da4986000865" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 45873 } }
I20260812 08:02:14.967590 18077 raft_consensus.cc:385] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:14.967658 18077 raft_consensus.cc:740] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 054c58374a914c0eabd9da4986000865, State: Initialized, Role: FOLLOWER
I20260812 08:02:14.967801 18075 consensus_queue.cc:260] T 0cf52873c58c4be6b624dec80436ef4c P f92587a8c94d4ac49cca93637e537626 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 } } peers { permanent_uuid: "c3f2977be270425889284dd0f2207502" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33501 } } peers { permanent_uuid: "054c58374a914c0eabd9da4986000865" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 45873 } }
I20260812 08:02:14.968075 18077 consensus_queue.cc:260] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 } } peers { permanent_uuid: "c3f2977be270425889284dd0f2207502" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33501 } } peers { permanent_uuid: "054c58374a914c0eabd9da4986000865" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 45873 } }
I20260812 08:02:14.968566 17593 heartbeater.cc:499] Master 127.5.142.190:37039 was elected leader, sending a full tablet report...
I20260812 08:02:14.968716 18075 ts_tablet_manager.cc:1434] T 0cf52873c58c4be6b624dec80436ef4c P f92587a8c94d4ac49cca93637e537626: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:02:14.968935 18077 ts_tablet_manager.cc:1434] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:02:14.968887 18076 raft_consensus.cc:359] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 } } peers { permanent_uuid: "c3f2977be270425889284dd0f2207502" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33501 } } peers { permanent_uuid: "054c58374a914c0eabd9da4986000865" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 45873 } }
I20260812 08:02:14.969120 18076 raft_consensus.cc:385] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:14.969198 18076 raft_consensus.cc:740] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3f2977be270425889284dd0f2207502, State: Initialized, Role: FOLLOWER
I20260812 08:02:14.969274 18049 heartbeater.cc:499] Master 127.5.142.190:37039 was elected leader, sending a full tablet report...
I20260812 08:02:14.969559 18076 consensus_queue.cc:260] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 } } peers { permanent_uuid: "c3f2977be270425889284dd0f2207502" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33501 } } peers { permanent_uuid: "054c58374a914c0eabd9da4986000865" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 45873 } }
I20260812 08:02:14.970126 17813 heartbeater.cc:499] Master 127.5.142.190:37039 was elected leader, sending a full tablet report...
I20260812 08:02:14.970350 18076 ts_tablet_manager.cc:1434] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
W20260812 08:02:15.006104 18050 tablet.cc:2406] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:02:15.030964 17814 tablet.cc:2406] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:02:15.058933 17595 tablet.cc:2406] T 0cf52873c58c4be6b624dec80436ef4c P f92587a8c94d4ac49cca93637e537626: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:02:15.121302 18086 raft_consensus.cc:493] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:02:15.121533 18086 raft_consensus.cc:515] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 } } peers { permanent_uuid: "c3f2977be270425889284dd0f2207502" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33501 } } peers { permanent_uuid: "054c58374a914c0eabd9da4986000865" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 45873 } }
I20260812 08:02:15.122844 18086 leader_election.cc:290] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f92587a8c94d4ac49cca93637e537626 (127.5.142.129:39477), 054c58374a914c0eabd9da4986000865 (127.5.142.131:45873)
I20260812 08:02:15.127585 17520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cf52873c58c4be6b624dec80436ef4c" candidate_uuid: "c3f2977be270425889284dd0f2207502" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f92587a8c94d4ac49cca93637e537626" is_pre_election: true
I20260812 08:02:15.127663 17961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cf52873c58c4be6b624dec80436ef4c" candidate_uuid: "c3f2977be270425889284dd0f2207502" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "054c58374a914c0eabd9da4986000865" is_pre_election: true
I20260812 08:02:15.127925 17961 raft_consensus.cc:2468] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3f2977be270425889284dd0f2207502 in term 0.
I20260812 08:02:15.127926 17520 raft_consensus.cc:2468] T 0cf52873c58c4be6b624dec80436ef4c P f92587a8c94d4ac49cca93637e537626 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3f2977be270425889284dd0f2207502 in term 0.
I20260812 08:02:15.128450 17628 leader_election.cc:304] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 054c58374a914c0eabd9da4986000865, c3f2977be270425889284dd0f2207502; no voters: 
I20260812 08:02:15.128746 18086 raft_consensus.cc:2804] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:02:15.128841 18086 raft_consensus.cc:493] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:02:15.128926 18086 raft_consensus.cc:3060] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:15.129895 18086 raft_consensus.cc:515] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 } } peers { permanent_uuid: "c3f2977be270425889284dd0f2207502" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33501 } } peers { permanent_uuid: "054c58374a914c0eabd9da4986000865" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 45873 } }
I20260812 08:02:15.130312 18086 leader_election.cc:290] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [CANDIDATE]: Term 1 election: Requested vote from peers f92587a8c94d4ac49cca93637e537626 (127.5.142.129:39477), 054c58374a914c0eabd9da4986000865 (127.5.142.131:45873)
I20260812 08:02:15.130724 17961 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cf52873c58c4be6b624dec80436ef4c" candidate_uuid: "c3f2977be270425889284dd0f2207502" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "054c58374a914c0eabd9da4986000865"
I20260812 08:02:15.130856 17961 raft_consensus.cc:3060] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:15.130836 17520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cf52873c58c4be6b624dec80436ef4c" candidate_uuid: "c3f2977be270425889284dd0f2207502" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f92587a8c94d4ac49cca93637e537626"
I20260812 08:02:15.131039 17520 raft_consensus.cc:3060] T 0cf52873c58c4be6b624dec80436ef4c P f92587a8c94d4ac49cca93637e537626 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:15.131906 17961 raft_consensus.cc:2468] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3f2977be270425889284dd0f2207502 in term 1.
I20260812 08:02:15.132262 17628 leader_election.cc:304] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 054c58374a914c0eabd9da4986000865, c3f2977be270425889284dd0f2207502; no voters: 
I20260812 08:02:15.132468 18086 raft_consensus.cc:2804] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:02:15.132483 17520 raft_consensus.cc:2468] T 0cf52873c58c4be6b624dec80436ef4c P f92587a8c94d4ac49cca93637e537626 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3f2977be270425889284dd0f2207502 in term 1.
I20260812 08:02:15.132751 18086 raft_consensus.cc:697] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [term 1 LEADER]: Becoming Leader. State: Replica: c3f2977be270425889284dd0f2207502, State: Running, Role: LEADER
I20260812 08:02:15.133162 18086 consensus_queue.cc:237] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 } } peers { permanent_uuid: "c3f2977be270425889284dd0f2207502" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33501 } } peers { permanent_uuid: "054c58374a914c0eabd9da4986000865" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 45873 } }
I20260812 08:02:15.136776 17284 catalog_manager.cc:5719] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 reported cstate change: term changed from 0 to 1, leader changed from <none> to c3f2977be270425889284dd0f2207502 (127.5.142.130). New cstate: current_term: 1 leader_uuid: "c3f2977be270425889284dd0f2207502" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3f2977be270425889284dd0f2207502" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "054c58374a914c0eabd9da4986000865" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 45873 } health_report { overall_health: UNKNOWN } } }
I20260812 08:02:15.182570  5690 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:02:15.184211  5690 ts_itest-base.cc:246] Waiting for 1 tablets on tserver f92587a8c94d4ac49cca93637e537626 to finish bootstrapping
I20260812 08:02:15.189523  5690 ts_itest-base.cc:246] Waiting for 1 tablets on tserver c3f2977be270425889284dd0f2207502 to finish bootstrapping
I20260812 08:02:15.194096  5690 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 054c58374a914c0eabd9da4986000865 to finish bootstrapping
I20260812 08:02:15.210072  5690 cluster_itest_util.cc:258] Not converged past 1 yet: 1.1 0.0 0.0
I20260812 08:02:15.312784  5690 cluster_itest_util.cc:258] Not converged past 1 yet: 1.1 0.0 0.0
I20260812 08:02:15.515733  5690 cluster_itest_util.cc:258] Not converged past 1 yet: 1.1 0.0 0.0
I20260812 08:02:15.516676 18086 consensus_queue.cc:1048] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:02:15.522536 18086 consensus_queue.cc:1048] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [LEADER]: Connected to new peer: Peer: permanent_uuid: "054c58374a914c0eabd9da4986000865" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 45873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:02:15.840337  5690 test_workload.cc:405] TestWorkload: Skipping table creation because table TestTable already exists
I20260812 08:02:15.840642  5690 raft_consensus-itest-base.cc:268] Waiting until we've written at least 4MB...
I20260812 08:02:15.926611  5690 raft_consensus-itest-base.cc:274] Waiting for log GC on c3f2977be270425889284dd0f2207502
I20260812 08:02:15.934712 17814 maintenance_manager.cc:419] P c3f2977be270425889284dd0f2207502: Scheduling FlushMRSOp(0cf52873c58c4be6b624dec80436ef4c): 4602202 bytes log retention, and flush 6119689 bytes memory
I20260812 08:02:15.963109 17595 maintenance_manager.cc:419] P f92587a8c94d4ac49cca93637e537626: Scheduling FlushMRSOp(0cf52873c58c4be6b624dec80436ef4c): 4207711 bytes log retention, and flush 6119689 bytes memory
I20260812 08:02:15.980945 17630 maintenance_manager.cc:643] P c3f2977be270425889284dd0f2207502: FlushMRSOp(0cf52873c58c4be6b624dec80436ef4c) complete. Timing: real 0.046s	user 0.046s	sys 0.000s Metrics: {"bytes_written":136001,"cfile_init":1,"compiler_manager_pool.queue_time_us":216,"drs_written":1,"lbm_read_time_us":73,"lbm_reads_lt_1ms":4,"lbm_write_time_us":673,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":36,"thread_start_us":240,"threads_started":2}
I20260812 08:02:15.981983 17814 maintenance_manager.cc:419] P c3f2977be270425889284dd0f2207502: Scheduling LogGCOp(0cf52873c58c4be6b624dec80436ef4c): free 2761328 bytes of WAL
I20260812 08:02:15.982268 17630 log_reader.cc:385] T 0cf52873c58c4be6b624dec80436ef4c: removed 3 log segments from log reader
I20260812 08:02:15.982337 17630 log.cc:1079] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502: Deleting log segment in path: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal/wals/0cf52873c58c4be6b624dec80436ef4c/wal-000000001 (ops 1-8)
I20260812 08:02:15.982407 17630 log.cc:1079] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502: Deleting log segment in path: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal/wals/0cf52873c58c4be6b624dec80436ef4c/wal-000000002 (ops 9-15)
I20260812 08:02:15.982450 17630 log.cc:1079] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502: Deleting log segment in path: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal/wals/0cf52873c58c4be6b624dec80436ef4c/wal-000000003 (ops 16-22)
I20260812 08:02:15.983428 17630 maintenance_manager.cc:643] P c3f2977be270425889284dd0f2207502: LogGCOp(0cf52873c58c4be6b624dec80436ef4c) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20260812 08:02:15.983738 17814 maintenance_manager.cc:419] P c3f2977be270425889284dd0f2207502: Scheduling UndoDeltaBlockGCOp(0cf52873c58c4be6b624dec80436ef4c): 535 bytes on disk
I20260812 08:02:15.984274 17630 maintenance_manager.cc:643] P c3f2977be270425889284dd0f2207502: UndoDeltaBlockGCOp(0cf52873c58c4be6b624dec80436ef4c) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":62,"lbm_reads_lt_1ms":4}
I20260812 08:02:15.988161  5690 raft_consensus-itest-base.cc:279] Log GC complete on c3f2977be270425889284dd0f2207502
I20260812 08:02:16.025367 17424 maintenance_manager.cc:643] P f92587a8c94d4ac49cca93637e537626: FlushMRSOp(0cf52873c58c4be6b624dec80436ef4c) complete. Timing: real 0.062s	user 0.057s	sys 0.004s Metrics: {"bytes_written":136001,"cfile_init":1,"compiler_manager_pool.queue_time_us":297,"dirs.queue_time_us":295,"dirs.run_cpu_time_us":226,"dirs.run_wall_time_us":774,"drs_written":1,"lbm_read_time_us":72,"lbm_reads_lt_1ms":4,"lbm_write_time_us":751,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":36,"thread_start_us":309,"threads_started":2}
I20260812 08:02:16.026492 17595 maintenance_manager.cc:419] P f92587a8c94d4ac49cca93637e537626: Scheduling LogGCOp(0cf52873c58c4be6b624dec80436ef4c): free 2629783 bytes of WAL
I20260812 08:02:16.026826 17424 log_reader.cc:385] T 0cf52873c58c4be6b624dec80436ef4c: removed 3 log segments from log reader
I20260812 08:02:16.026901 17424 log.cc:1079] T 0cf52873c58c4be6b624dec80436ef4c P f92587a8c94d4ac49cca93637e537626: Deleting log segment in path: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal/wals/0cf52873c58c4be6b624dec80436ef4c/wal-000000001 (ops 1-8)
I20260812 08:02:16.026961 17424 log.cc:1079] T 0cf52873c58c4be6b624dec80436ef4c P f92587a8c94d4ac49cca93637e537626: Deleting log segment in path: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal/wals/0cf52873c58c4be6b624dec80436ef4c/wal-000000002 (ops 9-14)
I20260812 08:02:16.026996 17424 log.cc:1079] T 0cf52873c58c4be6b624dec80436ef4c P f92587a8c94d4ac49cca93637e537626: Deleting log segment in path: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal/wals/0cf52873c58c4be6b624dec80436ef4c/wal-000000003 (ops 15-21)
I20260812 08:02:16.027941 17424 maintenance_manager.cc:643] P f92587a8c94d4ac49cca93637e537626: LogGCOp(0cf52873c58c4be6b624dec80436ef4c) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20260812 08:02:16.028250 17595 maintenance_manager.cc:419] P f92587a8c94d4ac49cca93637e537626: Scheduling UndoDeltaBlockGCOp(0cf52873c58c4be6b624dec80436ef4c): 535 bytes on disk
I20260812 08:02:16.028824 17424 maintenance_manager.cc:643] P f92587a8c94d4ac49cca93637e537626: UndoDeltaBlockGCOp(0cf52873c58c4be6b624dec80436ef4c) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":66,"lbm_reads_lt_1ms":4}
I20260812 08:02:17.989609  5690 raft_consensus-itest-base.cc:295] Servers converged with original term 1
I20260812 08:02:17.989683  5690 raft_consensus-itest-base.cc:300] Resuming 054c58374a914c0eabd9da4986000865
I20260812 08:02:17.991392 18099 raft_consensus.cc:493] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c3f2977be270425889284dd0f2207502)
I20260812 08:02:17.991515 18099 raft_consensus.cc:515] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 } } peers { permanent_uuid: "c3f2977be270425889284dd0f2207502" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33501 } } peers { permanent_uuid: "054c58374a914c0eabd9da4986000865" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 45873 } }
I20260812 08:02:17.992681 18099 leader_election.cc:290] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f92587a8c94d4ac49cca93637e537626 (127.5.142.129:39477), c3f2977be270425889284dd0f2207502 (127.5.142.130:33501)
I20260812 08:02:17.997731 18146 consensus_queue.cc:753] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [LEADER]: The logs necessary to catch up peer 054c58374a914c0eabd9da4986000865 have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 3..31: Segment 1 which contained index 3 has been GCed)
I20260812 08:02:17.997869 18146 consensus_queue.cc:579] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [LEADER]: The logs necessary to catch up peer 054c58374a914c0eabd9da4986000865 have been garbage collected. The replica will never be able to catch up
I20260812 08:02:17.999022 17739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cf52873c58c4be6b624dec80436ef4c" candidate_uuid: "054c58374a914c0eabd9da4986000865" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c3f2977be270425889284dd0f2207502" is_pre_election: true
I20260812 08:02:17.999195 17520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cf52873c58c4be6b624dec80436ef4c" candidate_uuid: "054c58374a914c0eabd9da4986000865" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f92587a8c94d4ac49cca93637e537626" is_pre_election: true
I20260812 08:02:17.999709 17845 leader_election.cc:304] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 054c58374a914c0eabd9da4986000865; no voters: c3f2977be270425889284dd0f2207502, f92587a8c94d4ac49cca93637e537626
I20260812 08:02:17.999928 18099 raft_consensus.cc:2749] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 08:02:18.001085 17739 consensus_queue.cc:237] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 37, Committed index: 37, Last appended: 1.37, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 } } peers { permanent_uuid: "c3f2977be270425889284dd0f2207502" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33501 } }
I20260812 08:02:18.002135 17520 raft_consensus.cc:1275] T 0cf52873c58c4be6b624dec80436ef4c P f92587a8c94d4ac49cca93637e537626 [term 1 FOLLOWER]: Refusing update from remote peer c3f2977be270425889284dd0f2207502: Log matching property violated. Preceding OpId in replica: term: 1 index: 37. Preceding OpId from leader: term: 1 index: 38. (index mismatch)
I20260812 08:02:18.002503 18163 consensus_queue.cc:1048] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 38, Last known committed idx: 37, Time since last communication: 0.000s
I20260812 08:02:18.004278 18146 raft_consensus.cc:2955] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [term 1 LEADER]: Committing config change with OpId 1.38: config changed from index -1 to 38, VOTER 054c58374a914c0eabd9da4986000865 (127.5.142.131) evicted. New config: { opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 } } peers { permanent_uuid: "c3f2977be270425889284dd0f2207502" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33501 } } }
I20260812 08:02:18.004664 17520 raft_consensus.cc:2955] T 0cf52873c58c4be6b624dec80436ef4c P f92587a8c94d4ac49cca93637e537626 [term 1 FOLLOWER]: Committing config change with OpId 1.38: config changed from index -1 to 38, VOTER 054c58374a914c0eabd9da4986000865 (127.5.142.131) evicted. New config: { opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 } } peers { permanent_uuid: "c3f2977be270425889284dd0f2207502" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33501 } } }
I20260812 08:02:18.006040 17263 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet 0cf52873c58c4be6b624dec80436ef4c with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 08:02:18.006918 17282 catalog_manager.cc:5719] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 reported cstate change: config changed from index -1 to 38, VOTER 054c58374a914c0eabd9da4986000865 (127.5.142.131) evicted. New cstate: current_term: 1 leader_uuid: "c3f2977be270425889284dd0f2207502" committed_config { opid_index: 38 OBSOLETE_local: false peers { permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3f2977be270425889284dd0f2207502" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33501 } health_report { overall_health: HEALTHY } } }
I20260812 08:02:18.010008 17929 tablet_service.cc:1558] Processing DeleteTablet for tablet 0cf52873c58c4be6b624dec80436ef4c with delete_type TABLET_DATA_TOMBSTONED (TS 054c58374a914c0eabd9da4986000865 not found in new config with opid_index 38) from {username='slave'} at 127.0.0.1:41216
I20260812 08:02:18.010564 18173 tablet_replica.cc:333] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865: stopping tablet replica
I20260812 08:02:18.010730 17739 consensus_queue.cc:237] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 38, Committed index: 38, Last appended: 1.38, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 } } peers { permanent_uuid: "c3f2977be270425889284dd0f2207502" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33501 } } peers { permanent_uuid: "054c58374a914c0eabd9da4986000865" member_type: NON_VOTER last_known_addr { host: "127.5.142.131" port: 45873 } attrs { promote: true } }
I20260812 08:02:18.010828 18173 raft_consensus.cc:2243] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 08:02:18.011049 18173 pending_rounds.cc:70] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865: Trying to abort 1 pending ops.
I20260812 08:02:18.012378 17520 raft_consensus.cc:1275] T 0cf52873c58c4be6b624dec80436ef4c P f92587a8c94d4ac49cca93637e537626 [term 1 FOLLOWER]: Refusing update from remote peer c3f2977be270425889284dd0f2207502: Log matching property violated. Preceding OpId in replica: term: 1 index: 38. Preceding OpId from leader: term: 1 index: 39. (index mismatch)
W20260812 08:02:18.012427 17628 consensus_peers.cc:597] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 -> Peer 054c58374a914c0eabd9da4986000865 (127.5.142.131:45873): Couldn't send request to peer 054c58374a914c0eabd9da4986000865. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:02:18.011147 18173 pending_rounds.cc:77] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7317593030049431552 op_type: WRITE_OP write_request { tablet_id: "0cf52873c58c4be6b624dec80436ef4c" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\000\300+<_\256^D\232\000\000\000\000\000\000\000\000\000\000\002\000\000\000\000\000" indirect_data: "<redacted>""000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
I20260812 08:02:18.012739 18163 consensus_queue.cc:1048] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 39, Last known committed idx: 38, Time since last communication: 0.000s
I20260812 08:02:18.013581 18173 raft_consensus.cc:2272] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 08:02:18.014333 18146 raft_consensus.cc:2955] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [term 1 LEADER]: Committing config change with OpId 1.39: config changed from index 38 to 39, NON_VOTER 054c58374a914c0eabd9da4986000865 (127.5.142.131) added. New config: { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 } } peers { permanent_uuid: "c3f2977be270425889284dd0f2207502" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33501 } } peers { permanent_uuid: "054c58374a914c0eabd9da4986000865" member_type: NON_VOTER last_known_addr { host: "127.5.142.131" port: 45873 } attrs { promote: true } } }
I20260812 08:02:18.014739 18173 ts_tablet_manager.cc:1916] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 08:02:18.014717 17520 raft_consensus.cc:2955] T 0cf52873c58c4be6b624dec80436ef4c P f92587a8c94d4ac49cca93637e537626 [term 1 FOLLOWER]: Committing config change with OpId 1.39: config changed from index 38 to 39, NON_VOTER 054c58374a914c0eabd9da4986000865 (127.5.142.131) added. New config: { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 } } peers { permanent_uuid: "c3f2977be270425889284dd0f2207502" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33501 } } peers { permanent_uuid: "054c58374a914c0eabd9da4986000865" member_type: NON_VOTER last_known_addr { host: "127.5.142.131" port: 45873 } attrs { promote: true } } }
I20260812 08:02:18.016328 17263 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 0cf52873c58c4be6b624dec80436ef4c with cas_config_opid_index 38: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 08:02:18.016842 17282 catalog_manager.cc:5719] T 0cf52873c58c4be6b624dec80436ef4c P f92587a8c94d4ac49cca93637e537626 reported cstate change: config changed from index 38 to 39, NON_VOTER 054c58374a914c0eabd9da4986000865 (127.5.142.131) added. New cstate: current_term: 1 leader_uuid: "c3f2977be270425889284dd0f2207502" committed_config { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 } } peers { permanent_uuid: "c3f2977be270425889284dd0f2207502" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33501 } } peers { permanent_uuid: "054c58374a914c0eabd9da4986000865" member_type: NON_VOTER last_known_addr { host: "127.5.142.131" port: 45873 } attrs { promote: true } } }
I20260812 08:02:18.017609 18173 ts_tablet_manager.cc:1929] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20260812 08:02:18.017700 18173 log.cc:1199] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865: Deleting WAL directory at /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal/wals/0cf52873c58c4be6b624dec80436ef4c
I20260812 08:02:18.018267 17264 catalog_manager.cc:5050] TS 054c58374a914c0eabd9da4986000865 (127.5.142.131:45873): tablet 0cf52873c58c4be6b624dec80436ef4c (table TestTable [id=21894f80755647b1aa91914be089f599]) successfully deleted
I20260812 08:02:18.983528 18176 ts_tablet_manager.cc:933] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865: Initiating tablet copy from peer c3f2977be270425889284dd0f2207502 (127.5.142.130:33501)
I20260812 08:02:18.984127 18176 tablet_copy_client.cc:323] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865: tablet copy: Beginning tablet copy session from remote peer at address 127.5.142.130:33501
I20260812 08:02:18.985018 17774 tablet_copy_service.cc:140] P c3f2977be270425889284dd0f2207502: Received BeginTabletCopySession request for tablet 0cf52873c58c4be6b624dec80436ef4c from peer 054c58374a914c0eabd9da4986000865 ({username='slave'} at 127.5.142.131:33229)
I20260812 08:02:18.985201 17774 tablet_copy_service.cc:161] P c3f2977be270425889284dd0f2207502: Beginning new tablet copy session on tablet 0cf52873c58c4be6b624dec80436ef4c from peer 054c58374a914c0eabd9da4986000865 at {username='slave'} at 127.5.142.131:33229: session id = 054c58374a914c0eabd9da4986000865-0cf52873c58c4be6b624dec80436ef4c
I20260812 08:02:18.986732 17774 tablet_copy_source_session.cc:215] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502: Tablet Copy: opened 5 blocks and 3 log segments
I20260812 08:02:18.988279 18176 ts_tablet_manager.cc:1916] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260812 08:02:18.990087 18176 ts_tablet_manager.cc:1929] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.2
I20260812 08:02:18.990275 18176 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cf52873c58c4be6b624dec80436ef4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:18.991459 18176 tablet_copy_client.cc:806] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865: tablet copy: Starting download of 5 data blocks...
I20260812 08:02:18.999320 18176 tablet_copy_client.cc:670] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865: tablet copy: Starting download of 3 WAL segments...
I20260812 08:02:19.008719 18176 tablet_copy_client.cc:538] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 08:02:19.010735 18176 tablet_bootstrap.cc:492] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865: Bootstrap starting.
I20260812 08:02:19.021379 18176 tablet_bootstrap.cc:492] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865: Bootstrap replayed 1/3 log segments. Stats: ops{read=7 overwritten=0 applied=4 ignored=4} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=2. Pending: 3 replicates
I20260812 08:02:19.025733 18176 tablet_bootstrap.cc:492] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865: Bootstrap replayed 2/3 log segments. Stats: ops{read=14 overwritten=0 applied=11 ignored=11} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=2. Pending: 3 replicates
I20260812 08:02:19.027845 18176 tablet_bootstrap.cc:492] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865: Bootstrap replayed 3/3 log segments. Stats: ops{read=17 overwritten=0 applied=17 ignored=15} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=2. Pending: 0 replicates
I20260812 08:02:19.028117 18176 tablet_bootstrap.cc:492] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865: Bootstrap complete.
I20260812 08:02:19.028661 18176 ts_tablet_manager.cc:1403] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865: Time spent bootstrapping tablet: real 0.018s	user 0.006s	sys 0.010s
I20260812 08:02:19.029233 18176 raft_consensus.cc:359] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 } } peers { permanent_uuid: "c3f2977be270425889284dd0f2207502" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33501 } } peers { permanent_uuid: "054c58374a914c0eabd9da4986000865" member_type: NON_VOTER last_known_addr { host: "127.5.142.131" port: 45873 } attrs { promote: true } }
I20260812 08:02:19.029340 18176 raft_consensus.cc:740] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 054c58374a914c0eabd9da4986000865, State: Initialized, Role: LEARNER
I20260812 08:02:19.029424 18176 consensus_queue.cc:260] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 39, Last appended: 1.39, Last appended by leader: 39, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 } } peers { permanent_uuid: "c3f2977be270425889284dd0f2207502" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33501 } } peers { permanent_uuid: "054c58374a914c0eabd9da4986000865" member_type: NON_VOTER last_known_addr { host: "127.5.142.131" port: 45873 } attrs { promote: true } }
I20260812 08:02:19.029753 18176 ts_tablet_manager.cc:1434] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865: Time spent starting tablet: real 0.001s	user 0.003s	sys 0.000s
I20260812 08:02:19.030468 17772 tablet_copy_service.cc:342] P c3f2977be270425889284dd0f2207502: Request end of tablet copy session 054c58374a914c0eabd9da4986000865-0cf52873c58c4be6b624dec80436ef4c received from {username='slave'} at 127.5.142.131:33229
I20260812 08:02:19.030583 17772 tablet_copy_service.cc:434] P c3f2977be270425889284dd0f2207502: ending tablet copy session 054c58374a914c0eabd9da4986000865-0cf52873c58c4be6b624dec80436ef4c on tablet 0cf52873c58c4be6b624dec80436ef4c with peer 054c58374a914c0eabd9da4986000865
I20260812 08:02:19.093847 18050 maintenance_manager.cc:419] P 054c58374a914c0eabd9da4986000865: Scheduling UndoDeltaBlockGCOp(0cf52873c58c4be6b624dec80436ef4c): 535 bytes on disk
I20260812 08:02:19.095108 17851 maintenance_manager.cc:643] P 054c58374a914c0eabd9da4986000865: UndoDeltaBlockGCOp(0cf52873c58c4be6b624dec80436ef4c) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":67,"lbm_reads_lt_1ms":4}
I20260812 08:02:19.456418 17961 raft_consensus.cc:1217] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.38->[1.39-1.39]   Dedup: 1.39->[]
I20260812 08:02:19.498195 17814 maintenance_manager.cc:419] P c3f2977be270425889284dd0f2207502: Scheduling LogGCOp(0cf52873c58c4be6b624dec80436ef4c): free 1840874 bytes of WAL
I20260812 08:02:19.498497 17630 log_reader.cc:385] T 0cf52873c58c4be6b624dec80436ef4c: removed 2 log segments from log reader
I20260812 08:02:19.498555 17630 log.cc:1079] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502: Deleting log segment in path: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal/wals/0cf52873c58c4be6b624dec80436ef4c/wal-000000004 (ops 23-29)
I20260812 08:02:19.498600 17630 log.cc:1079] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502: Deleting log segment in path: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-1/wal/wals/0cf52873c58c4be6b624dec80436ef4c/wal-000000005 (ops 30-36)
I20260812 08:02:19.499188 17630 maintenance_manager.cc:643] P c3f2977be270425889284dd0f2207502: LogGCOp(0cf52873c58c4be6b624dec80436ef4c) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20260812 08:02:19.996049 18191 raft_consensus.cc:1064] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502: attempting to promote NON_VOTER 054c58374a914c0eabd9da4986000865 to VOTER
I20260812 08:02:19.996538 18191 consensus_queue.cc:237] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 39, Committed index: 39, Last appended: 1.39, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 } } peers { permanent_uuid: "c3f2977be270425889284dd0f2207502" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33501 } } peers { permanent_uuid: "054c58374a914c0eabd9da4986000865" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 45873 } attrs { promote: false } }
I20260812 08:02:19.997869 17961 raft_consensus.cc:1275] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865 [term 1 LEARNER]: Refusing update from remote peer c3f2977be270425889284dd0f2207502: Log matching property violated. Preceding OpId in replica: term: 1 index: 39. Preceding OpId from leader: term: 1 index: 40. (index mismatch)
I20260812 08:02:19.997869 17520 raft_consensus.cc:1275] T 0cf52873c58c4be6b624dec80436ef4c P f92587a8c94d4ac49cca93637e537626 [term 1 FOLLOWER]: Refusing update from remote peer c3f2977be270425889284dd0f2207502: Log matching property violated. Preceding OpId in replica: term: 1 index: 39. Preceding OpId from leader: term: 1 index: 40. (index mismatch)
I20260812 08:02:19.998287 18192 consensus_queue.cc:1048] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 40, Last known committed idx: 39, Time since last communication: 0.000s
I20260812 08:02:19.998482 18191 consensus_queue.cc:1048] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [LEADER]: Connected to new peer: Peer: permanent_uuid: "054c58374a914c0eabd9da4986000865" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 45873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 40, Last known committed idx: 39, Time since last communication: 0.000s
I20260812 08:02:19.999320 18050 maintenance_manager.cc:419] P 054c58374a914c0eabd9da4986000865: Scheduling LogGCOp(0cf52873c58c4be6b624dec80436ef4c): free 1840820 bytes of WAL
I20260812 08:02:19.999527 17851 log_reader.cc:385] T 0cf52873c58c4be6b624dec80436ef4c: removed 2 log segments from log reader
I20260812 08:02:19.999595 17851 log.cc:1079] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865: Deleting log segment in path: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal/wals/0cf52873c58c4be6b624dec80436ef4c/wal-000000001 (ops 23-29)
I20260812 08:02:19.999642 17851 log.cc:1079] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865: Deleting log segment in path: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-2/wal/wals/0cf52873c58c4be6b624dec80436ef4c/wal-000000002 (ops 30-36)
I20260812 08:02:20.000373 17851 maintenance_manager.cc:643] P 054c58374a914c0eabd9da4986000865: LogGCOp(0cf52873c58c4be6b624dec80436ef4c) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20260812 08:02:20.000438 18192 raft_consensus.cc:2955] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 [term 1 LEADER]: Committing config change with OpId 1.40: config changed from index 39 to 40, 054c58374a914c0eabd9da4986000865 (127.5.142.131) changed from NON_VOTER to VOTER. New config: { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 } } peers { permanent_uuid: "c3f2977be270425889284dd0f2207502" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33501 } } peers { permanent_uuid: "054c58374a914c0eabd9da4986000865" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 45873 } attrs { promote: false } } }
I20260812 08:02:20.000862 17520 raft_consensus.cc:2955] T 0cf52873c58c4be6b624dec80436ef4c P f92587a8c94d4ac49cca93637e537626 [term 1 FOLLOWER]: Committing config change with OpId 1.40: config changed from index 39 to 40, 054c58374a914c0eabd9da4986000865 (127.5.142.131) changed from NON_VOTER to VOTER. New config: { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 } } peers { permanent_uuid: "c3f2977be270425889284dd0f2207502" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33501 } } peers { permanent_uuid: "054c58374a914c0eabd9da4986000865" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 45873 } attrs { promote: false } } }
I20260812 08:02:20.001343 17961 raft_consensus.cc:2955] T 0cf52873c58c4be6b624dec80436ef4c P 054c58374a914c0eabd9da4986000865 [term 1 FOLLOWER]: Committing config change with OpId 1.40: config changed from index 39 to 40, 054c58374a914c0eabd9da4986000865 (127.5.142.131) changed from NON_VOTER to VOTER. New config: { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 } } peers { permanent_uuid: "c3f2977be270425889284dd0f2207502" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33501 } } peers { permanent_uuid: "054c58374a914c0eabd9da4986000865" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 45873 } attrs { promote: false } } }
I20260812 08:02:20.003315 17281 catalog_manager.cc:5719] T 0cf52873c58c4be6b624dec80436ef4c P c3f2977be270425889284dd0f2207502 reported cstate change: config changed from index 39 to 40, 054c58374a914c0eabd9da4986000865 (127.5.142.131) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c3f2977be270425889284dd0f2207502" committed_config { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "f92587a8c94d4ac49cca93637e537626" member_type: VOTER last_known_addr { host: "127.5.142.129" port: 39477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3f2977be270425889284dd0f2207502" member_type: VOTER last_known_addr { host: "127.5.142.130" port: 33501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "054c58374a914c0eabd9da4986000865" member_type: VOTER last_known_addr { host: "127.5.142.131" port: 45873 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 08:02:20.548182 17595 maintenance_manager.cc:419] P f92587a8c94d4ac49cca93637e537626: Scheduling LogGCOp(0cf52873c58c4be6b624dec80436ef4c): free 1577928 bytes of WAL
I20260812 08:02:20.548534 17424 log_reader.cc:385] T 0cf52873c58c4be6b624dec80436ef4c: removed 2 log segments from log reader
I20260812 08:02:20.548602 17424 log.cc:1079] T 0cf52873c58c4be6b624dec80436ef4c P f92587a8c94d4ac49cca93637e537626: Deleting log segment in path: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal/wals/0cf52873c58c4be6b624dec80436ef4c/wal-000000004 (ops 22-26)
I20260812 08:02:20.548650 17424 log.cc:1079] T 0cf52873c58c4be6b624dec80436ef4c P f92587a8c94d4ac49cca93637e537626: Deleting log segment in path: /tmp/dist-test-taskapknlq/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786521651263816-5690-0/raft_consensus-itest-cluster/ts-0/wal/wals/0cf52873c58c4be6b624dec80436ef4c/wal-000000005 (ops 27-33)
I20260812 08:02:20.549260 17424 maintenance_manager.cc:643] P f92587a8c94d4ac49cca93637e537626: LogGCOp(0cf52873c58c4be6b624dec80436ef4c) complete. Timing: real 0.001s	user 0.000s	sys 0.001s Metrics: {}
I20260812 08:02:23.001158  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 17398
I20260812 08:02:23.018010  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 17600
I20260812 08:02:23.034413  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 17816
I20260812 08:02:23.048208  5690 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskapknlq/build/debug/bin/kudu with pid 17237
2026-08-12T08:02:23Z chronyd exiting
[       OK ] EncryptionEnabled/RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC/0 (8965 ms)
[----------] 1 test from EncryptionEnabled/RaftConsensusParamEncryptionITest (8965 ms total)

[----------] Global test environment tear-down
[==========] 7 tests from 2 test suites ran. (91801 ms total)
[  PASSED  ] 7 tests.
I20260812 08:02:23.077507  5690 logging.cc:436] LogThrottler /workspace/apache/dev/local/kudu/src/kudu/client/batcher.cc:441: suppressed but not reported on 11 messages since previous log ~14 seconds ago
