Note: This is test shard 3 of 6.
[==========] Running 4 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 3 tests from TabletCopyITest
[ RUN      ] TabletCopyITest.TestCopyAfterFailedCopy
2026-08-12T08:03:33Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:03:33Z Disabled control of system clock
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260812 08:03:33.157058 15590 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
/tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/master-0/data-0,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/master-0/data-1
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/master-0/data-0/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.57.190:44827
--webserver_interface=127.15.57.190
--webserver_port=0
--builtin_ntp_servers=127.15.57.148:33175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.57.190:44827
--master_tombstone_evicted_tablet_replicas=false with env {}
W20260812 08:03:33.301057 15605 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:33.301409 15605 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:33.301457 15605 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:33.305200 15605 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:03:33.305284 15605 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:33.305316 15605 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:03:33.305341 15605 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:03:33.310117 15605 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.57.148:33175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/master-0/data-0,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/master-0/data-1
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/master-0/wal
--master_tombstone_evicted_tablet_replicas=false
--ipki_ca_key_size=768
--master_addresses=127.15.57.190:44827
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.57.190:44827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/master-0/data-0/info.pb
--webserver_interface=127.15.57.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.15605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:33.311847 15605 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:33.313545 15605 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260812 08:03:33.321576 15605 server_base.cc:1061] running on GCE node
W20260812 08:03:33.321485 15616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:33.321485 15620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:33.321485 15615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:33.322625 15605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:33.323930 15605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:33.325203 15605 hybrid_clock.cc:648] HybridClock initialized: now 1786521813325157 us; error 65 us; skew 500 ppm
I20260812 08:03:33.328008 15605 webserver.cc:533] Webserver started at http://127.15.57.190:39365/ using document root <none> and password file <none>
I20260812 08:03:33.328923 15605 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:33.328992 15605 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:33.329284 15605 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:33.331364 15605 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/master-0/data-0/instance:
uuid: "ce5d17840253409194189eb97ebf2481"
format_stamp: "Formatted at 2026-08-12 08:03:33 on dist-test-slave-h3ft"
I20260812 08:03:33.331943 15605 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/master-0/data-1/instance:
uuid: "ce5d17840253409194189eb97ebf2481"
format_stamp: "Formatted at 2026-08-12 08:03:33 on dist-test-slave-h3ft"
I20260812 08:03:33.332252 15605 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/master-0/wal/instance:
uuid: "ce5d17840253409194189eb97ebf2481"
format_stamp: "Formatted at 2026-08-12 08:03:33 on dist-test-slave-h3ft"
I20260812 08:03:33.337622 15605 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.002s
I20260812 08:03:33.340782 15629 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:33.340785 15628 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:33.342483 15605 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260812 08:03:33.342661 15605 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/master-0/data-0,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/master-0/data-1,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/master-0/wal
uuid: "ce5d17840253409194189eb97ebf2481"
format_stamp: "Formatted at 2026-08-12 08:03:33 on dist-test-slave-h3ft"
I20260812 08:03:33.342796 15605 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/master-0/wal
2 data directories: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/master-0/data-0/data, /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/master-0/data-1/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:33.363788 15605 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:33.364828 15605 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:33.365141 15605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:33.375437 15605 rpc_server.cc:307] RPC server started. Bound to: 127.15.57.190:44827
I20260812 08:03:33.375491 15721 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.57.190:44827 every 8 connection(s)
I20260812 08:03:33.377079 15605 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/master-0/data-0/info.pb
I20260812 08:03:33.381906 15723 data_dirs.cc:400] Could only allocate 2 dirs of requested 3 for tablet 00000000000000000000000000000000. 2 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:33.386845 15590 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu as pid 15605
I20260812 08:03:33.387151 15590 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/master-0/wal/instance
I20260812 08:03:33.390034 15723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce5d17840253409194189eb97ebf2481: Bootstrap starting.
I20260812 08:03:33.393517 15723 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce5d17840253409194189eb97ebf2481: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:33.394595 15723 log.cc:826] T 00000000000000000000000000000000 P ce5d17840253409194189eb97ebf2481: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:33.397086 15723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce5d17840253409194189eb97ebf2481: No bootstrap required, opened a new log
I20260812 08:03:33.400976 15723 raft_consensus.cc:359] T 00000000000000000000000000000000 P ce5d17840253409194189eb97ebf2481 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5d17840253409194189eb97ebf2481" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 44827 } }
I20260812 08:03:33.401228 15723 raft_consensus.cc:385] T 00000000000000000000000000000000 P ce5d17840253409194189eb97ebf2481 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:33.401302 15723 raft_consensus.cc:740] T 00000000000000000000000000000000 P ce5d17840253409194189eb97ebf2481 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce5d17840253409194189eb97ebf2481, State: Initialized, Role: FOLLOWER
I20260812 08:03:33.401944 15723 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce5d17840253409194189eb97ebf2481 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5d17840253409194189eb97ebf2481" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 44827 } }
I20260812 08:03:33.402153 15723 raft_consensus.cc:399] T 00000000000000000000000000000000 P ce5d17840253409194189eb97ebf2481 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:03:33.402225 15723 raft_consensus.cc:493] T 00000000000000000000000000000000 P ce5d17840253409194189eb97ebf2481 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:03:33.402338 15723 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ce5d17840253409194189eb97ebf2481 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:33.403669 15723 raft_consensus.cc:515] T 00000000000000000000000000000000 P ce5d17840253409194189eb97ebf2481 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5d17840253409194189eb97ebf2481" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 44827 } }
I20260812 08:03:33.404168 15723 leader_election.cc:304] T 00000000000000000000000000000000 P ce5d17840253409194189eb97ebf2481 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce5d17840253409194189eb97ebf2481; no voters: 
I20260812 08:03:33.404685 15723 leader_election.cc:290] T 00000000000000000000000000000000 P ce5d17840253409194189eb97ebf2481 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:03:33.404851 15729 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ce5d17840253409194189eb97ebf2481 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:33.405157 15729 raft_consensus.cc:697] T 00000000000000000000000000000000 P ce5d17840253409194189eb97ebf2481 [term 1 LEADER]: Becoming Leader. State: Replica: ce5d17840253409194189eb97ebf2481, State: Running, Role: LEADER
I20260812 08:03:33.405581 15729 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce5d17840253409194189eb97ebf2481 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5d17840253409194189eb97ebf2481" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 44827 } }
I20260812 08:03:33.406240 15723 sys_catalog.cc:565] T 00000000000000000000000000000000 P ce5d17840253409194189eb97ebf2481 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:03:33.407270 15732 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce5d17840253409194189eb97ebf2481 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce5d17840253409194189eb97ebf2481. Latest consensus state: current_term: 1 leader_uuid: "ce5d17840253409194189eb97ebf2481" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5d17840253409194189eb97ebf2481" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 44827 } } }
I20260812 08:03:33.407510 15732 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce5d17840253409194189eb97ebf2481 [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:33.407895 15736 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce5d17840253409194189eb97ebf2481 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce5d17840253409194189eb97ebf2481" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5d17840253409194189eb97ebf2481" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 44827 } } }
I20260812 08:03:33.408082 15736 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce5d17840253409194189eb97ebf2481 [sys.catalog]: This master's current role is: LEADER
W20260812 08:03:33.412452 15752 catalog_manager.cc:1594] T 00000000000000000000000000000000 P ce5d17840253409194189eb97ebf2481: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 08:03:33.412534 15752 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 08:03:33.412636 15744 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:03:33.413635 15744 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:03:33.419930 15744 catalog_manager.cc:1383] Generated new cluster ID: 0e9c3b6fdea24697b2fd996aad206cde
I20260812 08:03:33.420089 15744 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:03:33.431156 15744 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:03:33.432510 15744 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:03:33.438975 15744 catalog_manager.cc:6092] T 00000000000000000000000000000000 P ce5d17840253409194189eb97ebf2481: Generated new TSK 0
I20260812 08:03:33.439780 15744 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:03:33.444836 15590 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
/tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-0/data-0,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-0/data-1
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-0/data-0/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.57.129:0
--local_ip_for_outbound_sockets=127.15.57.129
--webserver_interface=127.15.57.129
--webserver_port=0
--tserver_master_addrs=127.15.57.190:44827
--builtin_ntp_servers=127.15.57.148:33175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20260812 08:03:33.583411 15758 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:33.583762 15758 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:33.583849 15758 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:33.587646 15758 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:33.587867 15758 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.57.129
I20260812 08:03:33.592525 15758 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.57.148:33175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-0/data-0,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-0/data-1
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.57.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-0/data-0/info.pb
--webserver_interface=127.15.57.129
--webserver_port=0
--tserver_master_addrs=127.15.57.190:44827
--never_fsync=true
--heap_profile_path=/tmp/kudu.15758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.57.129

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:33.594148 15758 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:33.595907 15758 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:33.604964 15773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:33.605039 15776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:33.604964 15772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:33.605198 15758 server_base.cc:1061] running on GCE node
I20260812 08:03:33.605816 15758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:33.606489 15758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:33.607652 15758 hybrid_clock.cc:648] HybridClock initialized: now 1786521813607633 us; error 34 us; skew 500 ppm
I20260812 08:03:33.610158 15758 webserver.cc:533] Webserver started at http://127.15.57.129:39613/ using document root <none> and password file <none>
I20260812 08:03:33.611076 15758 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:33.611152 15758 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:33.611429 15758 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:33.613485 15758 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-0/data-0/instance:
uuid: "a720c9521091492f8807b8e38a40e773"
format_stamp: "Formatted at 2026-08-12 08:03:33 on dist-test-slave-h3ft"
I20260812 08:03:33.614075 15758 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-0/data-1/instance:
uuid: "a720c9521091492f8807b8e38a40e773"
format_stamp: "Formatted at 2026-08-12 08:03:33 on dist-test-slave-h3ft"
I20260812 08:03:33.614414 15758 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-0/wal/instance:
uuid: "a720c9521091492f8807b8e38a40e773"
format_stamp: "Formatted at 2026-08-12 08:03:33 on dist-test-slave-h3ft"
I20260812 08:03:33.619571 15758 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.001s
I20260812 08:03:33.622443 15786 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:33.622462 15785 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:33.624035 15758 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:03:33.624186 15758 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-0/data-0,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-0/data-1,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-0/wal
uuid: "a720c9521091492f8807b8e38a40e773"
format_stamp: "Formatted at 2026-08-12 08:03:33 on dist-test-slave-h3ft"
I20260812 08:03:33.624303 15758 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-0/wal
2 data directories: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-0/data-0/data, /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-0/data-1/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:33.635447 15758 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:33.636318 15758 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:33.636574 15758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:33.637326 15758 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:33.638679 15758 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:33.638756 15758 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:33.638833 15758 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:33.638867 15758 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:33.651352 15758 rpc_server.cc:307] RPC server started. Bound to: 127.15.57.129:38967
I20260812 08:03:33.651404 15962 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.57.129:38967 every 8 connection(s)
I20260812 08:03:33.652711 15758 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-0/data-0/info.pb
I20260812 08:03:33.655339 15590 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu as pid 15758
I20260812 08:03:33.655467 15590 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-0/wal/instance
I20260812 08:03:33.658859 15590 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
/tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-1/data-0,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-1/data-1
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-1/data-0/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.57.130:0
--local_ip_for_outbound_sockets=127.15.57.130
--webserver_interface=127.15.57.130
--webserver_port=0
--tserver_master_addrs=127.15.57.190:44827
--builtin_ntp_servers=127.15.57.148:33175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260812 08:03:33.666201 15963 heartbeater.cc:344] Connected to a master server at 127.15.57.190:44827
I20260812 08:03:33.666637 15963 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:33.667505 15963 heartbeater.cc:507] Master 127.15.57.190:44827 requested a full tablet report, sending...
I20260812 08:03:33.670138 15660 ts_manager.cc:194] Registered new tserver with Master: a720c9521091492f8807b8e38a40e773 (127.15.57.129:38967)
I20260812 08:03:33.672418 15660 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.15.57.129:53247
W20260812 08:03:33.800545 15974 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:33.800909 15974 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:33.800966 15974 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:33.804997 15974 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:33.805135 15974 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.57.130
I20260812 08:03:33.809710 15974 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.57.148:33175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-1/data-0,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-1/data-1
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.57.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-1/data-0/info.pb
--webserver_interface=127.15.57.130
--webserver_port=0
--tserver_master_addrs=127.15.57.190:44827
--never_fsync=true
--heap_profile_path=/tmp/kudu.15974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.57.130

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:33.811421 15974 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:33.813014 15974 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:33.822129 15983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:33.822129 15986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:33.822259 15980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:33.822225 15974 server_base.cc:1061] running on GCE node
I20260812 08:03:33.822993 15974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:33.823747 15974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:33.824966 15974 hybrid_clock.cc:648] HybridClock initialized: now 1786521813824945 us; error 42 us; skew 500 ppm
I20260812 08:03:33.827559 15974 webserver.cc:533] Webserver started at http://127.15.57.130:42405/ using document root <none> and password file <none>
I20260812 08:03:33.828401 15974 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:33.828467 15974 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:33.828737 15974 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:33.830883 15974 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-1/data-0/instance:
uuid: "a49698551c7249ae87856561a708cf10"
format_stamp: "Formatted at 2026-08-12 08:03:33 on dist-test-slave-h3ft"
I20260812 08:03:33.831564 15974 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-1/data-1/instance:
uuid: "a49698551c7249ae87856561a708cf10"
format_stamp: "Formatted at 2026-08-12 08:03:33 on dist-test-slave-h3ft"
I20260812 08:03:33.831897 15974 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-1/wal/instance:
uuid: "a49698551c7249ae87856561a708cf10"
format_stamp: "Formatted at 2026-08-12 08:03:33 on dist-test-slave-h3ft"
I20260812 08:03:33.837502 15974 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260812 08:03:33.840890 15999 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:33.840893 15997 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:33.842705 15974 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260812 08:03:33.842883 15974 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-1/data-0,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-1/data-1,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-1/wal
uuid: "a49698551c7249ae87856561a708cf10"
format_stamp: "Formatted at 2026-08-12 08:03:33 on dist-test-slave-h3ft"
I20260812 08:03:33.843096 15974 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-1/wal
2 data directories: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-1/data-0/data, /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-1/data-1/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:33.860055 15974 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:33.861076 15974 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:33.861343 15974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:33.862197 15974 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:33.863814 15974 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:33.863873 15974 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:33.863924 15974 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:33.863981 15974 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:33.876103 15974 rpc_server.cc:307] RPC server started. Bound to: 127.15.57.130:34467
I20260812 08:03:33.876127 16189 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.57.130:34467 every 8 connection(s)
I20260812 08:03:33.877372 15974 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-1/data-0/info.pb
I20260812 08:03:33.881130 15590 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu as pid 15974
I20260812 08:03:33.881278 15590 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-1/wal/instance
I20260812 08:03:33.884146 15590 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
/tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-2/data-0,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-2/data-1
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-2/data-0/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.57.131:0
--local_ip_for_outbound_sockets=127.15.57.131
--webserver_interface=127.15.57.131
--webserver_port=0
--tserver_master_addrs=127.15.57.190:44827
--builtin_ntp_servers=127.15.57.148:33175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260812 08:03:33.888870 16190 heartbeater.cc:344] Connected to a master server at 127.15.57.190:44827
I20260812 08:03:33.889312 16190 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:33.890280 16190 heartbeater.cc:507] Master 127.15.57.190:44827 requested a full tablet report, sending...
I20260812 08:03:33.892136 15660 ts_manager.cc:194] Registered new tserver with Master: a49698551c7249ae87856561a708cf10 (127.15.57.130:34467)
I20260812 08:03:33.892959 15660 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.15.57.130:53765
W20260812 08:03:34.022430 16194 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:34.022755 16194 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:34.022845 16194 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:34.026453 16194 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:34.026628 16194 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.57.131
I20260812 08:03:34.030753 16194 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.57.148:33175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-2/data-0,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-2/data-1
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.57.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-2/data-0/info.pb
--webserver_interface=127.15.57.131
--webserver_port=0
--tserver_master_addrs=127.15.57.190:44827
--never_fsync=true
--heap_profile_path=/tmp/kudu.16194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.57.131

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:34.032305 16194 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:34.033846 16194 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:34.042099 16206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:34.042104 16203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:34.042351 16194 server_base.cc:1061] running on GCE node
W20260812 08:03:34.042099 16201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:34.042811 16194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:34.043591 16194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:34.044785 16194 hybrid_clock.cc:648] HybridClock initialized: now 1786521814044771 us; error 39 us; skew 500 ppm
I20260812 08:03:34.047209 16194 webserver.cc:533] Webserver started at http://127.15.57.131:40209/ using document root <none> and password file <none>
I20260812 08:03:34.048215 16194 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:34.048336 16194 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:34.048732 16194 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:34.050834 16194 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-2/data-0/instance:
uuid: "a464d70ec5fb4cd0b393cc467d4e9fad"
format_stamp: "Formatted at 2026-08-12 08:03:34 on dist-test-slave-h3ft"
I20260812 08:03:34.051550 16194 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-2/data-1/instance:
uuid: "a464d70ec5fb4cd0b393cc467d4e9fad"
format_stamp: "Formatted at 2026-08-12 08:03:34 on dist-test-slave-h3ft"
I20260812 08:03:34.051913 16194 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-2/wal/instance:
uuid: "a464d70ec5fb4cd0b393cc467d4e9fad"
format_stamp: "Formatted at 2026-08-12 08:03:34 on dist-test-slave-h3ft"
I20260812 08:03:34.057739 16194 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260812 08:03:34.061787 16224 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:34.061787 16222 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:34.064316 16194 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.000s
I20260812 08:03:34.064548 16194 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-2/data-0,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-2/data-1,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-2/wal
uuid: "a464d70ec5fb4cd0b393cc467d4e9fad"
format_stamp: "Formatted at 2026-08-12 08:03:34 on dist-test-slave-h3ft"
I20260812 08:03:34.064719 16194 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-2/wal
2 data directories: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-2/data-0/data, /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-2/data-1/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:34.089502 16194 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:34.090619 16194 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:34.091104 16194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:34.091990 16194 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:34.093544 16194 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:34.093631 16194 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:34.093719 16194 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:34.093767 16194 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:34.106935 16194 rpc_server.cc:307] RPC server started. Bound to: 127.15.57.131:34751
I20260812 08:03:34.106951 16397 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.57.131:34751 every 8 connection(s)
I20260812 08:03:34.108665 16194 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-2/data-0/info.pb
I20260812 08:03:34.117648 15590 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu as pid 16194
I20260812 08:03:34.117821 15590 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-2/wal/instance
I20260812 08:03:34.121786 16398 heartbeater.cc:344] Connected to a master server at 127.15.57.190:44827
I20260812 08:03:34.122188 16398 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:34.122988 16398 heartbeater.cc:507] Master 127.15.57.190:44827 requested a full tablet report, sending...
I20260812 08:03:34.124607 15660 ts_manager.cc:194] Registered new tserver with Master: a464d70ec5fb4cd0b393cc467d4e9fad (127.15.57.131:34751)
I20260812 08:03:34.125367 15660 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.15.57.131:59261
I20260812 08:03:34.134904 15590 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:03:34.165684 15660 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48042:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260812 08:03:34.167400 15660 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 08:03:34.188279 15857 tablet_service.cc:1511] Processing CreateTablet for tablet c0f3091b3b2d45dc9ac7d52e2d60f5d5 (DEFAULT_TABLE table=test-workload [id=a7e13a632bc2424cadda09f87e1d3c5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:34.188288 16284 tablet_service.cc:1511] Processing CreateTablet for tablet c0f3091b3b2d45dc9ac7d52e2d60f5d5 (DEFAULT_TABLE table=test-workload [id=a7e13a632bc2424cadda09f87e1d3c5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:34.188700 16070 tablet_service.cc:1511] Processing CreateTablet for tablet c0f3091b3b2d45dc9ac7d52e2d60f5d5 (DEFAULT_TABLE table=test-workload [id=a7e13a632bc2424cadda09f87e1d3c5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:34.189566 15857 data_dirs.cc:400] Could only allocate 2 dirs of requested 3 for tablet c0f3091b3b2d45dc9ac7d52e2d60f5d5. 2 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:34.189569 16284 data_dirs.cc:400] Could only allocate 2 dirs of requested 3 for tablet c0f3091b3b2d45dc9ac7d52e2d60f5d5. 2 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:34.189636 16070 data_dirs.cc:400] Could only allocate 2 dirs of requested 3 for tablet c0f3091b3b2d45dc9ac7d52e2d60f5d5. 2 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:34.196174 16430 tablet_bootstrap.cc:492] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a720c9521091492f8807b8e38a40e773: Bootstrap starting.
I20260812 08:03:34.196853 16432 tablet_bootstrap.cc:492] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a464d70ec5fb4cd0b393cc467d4e9fad: Bootstrap starting.
I20260812 08:03:34.196894 16431 tablet_bootstrap.cc:492] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10: Bootstrap starting.
I20260812 08:03:34.198751 16430 tablet_bootstrap.cc:654] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a720c9521091492f8807b8e38a40e773: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:34.199221 16431 tablet_bootstrap.cc:654] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:34.199221 16432 tablet_bootstrap.cc:654] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a464d70ec5fb4cd0b393cc467d4e9fad: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:34.199796 16430 log.cc:826] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a720c9521091492f8807b8e38a40e773: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:34.200111 16432 log.cc:826] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a464d70ec5fb4cd0b393cc467d4e9fad: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:34.200111 16431 log.cc:826] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:34.201758 16432 tablet_bootstrap.cc:492] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a464d70ec5fb4cd0b393cc467d4e9fad: No bootstrap required, opened a new log
I20260812 08:03:34.201766 16430 tablet_bootstrap.cc:492] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a720c9521091492f8807b8e38a40e773: No bootstrap required, opened a new log
I20260812 08:03:34.202006 16430 ts_tablet_manager.cc:1403] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a720c9521091492f8807b8e38a40e773: Time spent bootstrapping tablet: real 0.006s	user 0.002s	sys 0.002s
I20260812 08:03:34.202041 16432 ts_tablet_manager.cc:1403] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a464d70ec5fb4cd0b393cc467d4e9fad: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:03:34.203253 16431 tablet_bootstrap.cc:492] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10: No bootstrap required, opened a new log
I20260812 08:03:34.203460 16431 ts_tablet_manager.cc:1403] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10: Time spent bootstrapping tablet: real 0.007s	user 0.005s	sys 0.000s
I20260812 08:03:34.205137 16432 raft_consensus.cc:359] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a464d70ec5fb4cd0b393cc467d4e9fad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a464d70ec5fb4cd0b393cc467d4e9fad" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 34751 } } peers { permanent_uuid: "a49698551c7249ae87856561a708cf10" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 34467 } } peers { permanent_uuid: "a720c9521091492f8807b8e38a40e773" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 38967 } }
I20260812 08:03:34.205372 16432 raft_consensus.cc:385] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a464d70ec5fb4cd0b393cc467d4e9fad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:34.205461 16432 raft_consensus.cc:740] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a464d70ec5fb4cd0b393cc467d4e9fad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a464d70ec5fb4cd0b393cc467d4e9fad, State: Initialized, Role: FOLLOWER
I20260812 08:03:34.205441 16430 raft_consensus.cc:359] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a720c9521091492f8807b8e38a40e773 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a464d70ec5fb4cd0b393cc467d4e9fad" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 34751 } } peers { permanent_uuid: "a49698551c7249ae87856561a708cf10" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 34467 } } peers { permanent_uuid: "a720c9521091492f8807b8e38a40e773" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 38967 } }
I20260812 08:03:34.205659 16430 raft_consensus.cc:385] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a720c9521091492f8807b8e38a40e773 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:34.205718 16430 raft_consensus.cc:740] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a720c9521091492f8807b8e38a40e773 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a720c9521091492f8807b8e38a40e773, State: Initialized, Role: FOLLOWER
I20260812 08:03:34.206125 16432 consensus_queue.cc:260] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a464d70ec5fb4cd0b393cc467d4e9fad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a464d70ec5fb4cd0b393cc467d4e9fad" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 34751 } } peers { permanent_uuid: "a49698551c7249ae87856561a708cf10" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 34467 } } peers { permanent_uuid: "a720c9521091492f8807b8e38a40e773" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 38967 } }
I20260812 08:03:34.206169 16430 consensus_queue.cc:260] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a720c9521091492f8807b8e38a40e773 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a464d70ec5fb4cd0b393cc467d4e9fad" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 34751 } } peers { permanent_uuid: "a49698551c7249ae87856561a708cf10" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 34467 } } peers { permanent_uuid: "a720c9521091492f8807b8e38a40e773" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 38967 } }
I20260812 08:03:34.206655 16431 raft_consensus.cc:359] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a464d70ec5fb4cd0b393cc467d4e9fad" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 34751 } } peers { permanent_uuid: "a49698551c7249ae87856561a708cf10" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 34467 } } peers { permanent_uuid: "a720c9521091492f8807b8e38a40e773" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 38967 } }
I20260812 08:03:34.206919 16431 raft_consensus.cc:385] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:34.206924 16398 heartbeater.cc:499] Master 127.15.57.190:44827 was elected leader, sending a full tablet report...
I20260812 08:03:34.207057 16431 raft_consensus.cc:740] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a49698551c7249ae87856561a708cf10, State: Initialized, Role: FOLLOWER
I20260812 08:03:34.207289 16432 ts_tablet_manager.cc:1434] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a464d70ec5fb4cd0b393cc467d4e9fad: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260812 08:03:34.207288 16430 ts_tablet_manager.cc:1434] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a720c9521091492f8807b8e38a40e773: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260812 08:03:34.207722 15963 heartbeater.cc:499] Master 127.15.57.190:44827 was elected leader, sending a full tablet report...
I20260812 08:03:34.207686 16431 consensus_queue.cc:260] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a464d70ec5fb4cd0b393cc467d4e9fad" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 34751 } } peers { permanent_uuid: "a49698551c7249ae87856561a708cf10" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 34467 } } peers { permanent_uuid: "a720c9521091492f8807b8e38a40e773" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 38967 } }
I20260812 08:03:34.208561 16431 ts_tablet_manager.cc:1434] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260812 08:03:34.208796 16190 heartbeater.cc:499] Master 127.15.57.190:44827 was elected leader, sending a full tablet report...
I20260812 08:03:34.220564 16440 raft_consensus.cc:493] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a720c9521091492f8807b8e38a40e773 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:34.220831 16440 raft_consensus.cc:515] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a720c9521091492f8807b8e38a40e773 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a464d70ec5fb4cd0b393cc467d4e9fad" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 34751 } } peers { permanent_uuid: "a49698551c7249ae87856561a708cf10" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 34467 } } peers { permanent_uuid: "a720c9521091492f8807b8e38a40e773" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 38967 } }
I20260812 08:03:34.222389 16440 leader_election.cc:290] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a720c9521091492f8807b8e38a40e773 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a464d70ec5fb4cd0b393cc467d4e9fad (127.15.57.131:34751), a49698551c7249ae87856561a708cf10 (127.15.57.130:34467)
I20260812 08:03:34.227859 16103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f3091b3b2d45dc9ac7d52e2d60f5d5" candidate_uuid: "a720c9521091492f8807b8e38a40e773" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a49698551c7249ae87856561a708cf10" is_pre_election: true
I20260812 08:03:34.227945 16325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f3091b3b2d45dc9ac7d52e2d60f5d5" candidate_uuid: "a720c9521091492f8807b8e38a40e773" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a464d70ec5fb4cd0b393cc467d4e9fad" is_pre_election: true
I20260812 08:03:34.228304 16103 raft_consensus.cc:2468] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720c9521091492f8807b8e38a40e773 in term 0.
I20260812 08:03:34.228310 16325 raft_consensus.cc:2468] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a464d70ec5fb4cd0b393cc467d4e9fad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a720c9521091492f8807b8e38a40e773 in term 0.
I20260812 08:03:34.228914 15788 leader_election.cc:304] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a720c9521091492f8807b8e38a40e773 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a464d70ec5fb4cd0b393cc467d4e9fad, a720c9521091492f8807b8e38a40e773; no voters: 
I20260812 08:03:34.229267 16440 raft_consensus.cc:2804] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a720c9521091492f8807b8e38a40e773 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:03:34.229367 16440 raft_consensus.cc:493] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a720c9521091492f8807b8e38a40e773 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:03:34.229418 16440 raft_consensus.cc:3060] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a720c9521091492f8807b8e38a40e773 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:34.230551 16440 raft_consensus.cc:515] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a720c9521091492f8807b8e38a40e773 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a464d70ec5fb4cd0b393cc467d4e9fad" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 34751 } } peers { permanent_uuid: "a49698551c7249ae87856561a708cf10" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 34467 } } peers { permanent_uuid: "a720c9521091492f8807b8e38a40e773" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 38967 } }
I20260812 08:03:34.231040 16440 leader_election.cc:290] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a720c9521091492f8807b8e38a40e773 [CANDIDATE]: Term 1 election: Requested vote from peers a464d70ec5fb4cd0b393cc467d4e9fad (127.15.57.131:34751), a49698551c7249ae87856561a708cf10 (127.15.57.130:34467)
I20260812 08:03:34.231469 16325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f3091b3b2d45dc9ac7d52e2d60f5d5" candidate_uuid: "a720c9521091492f8807b8e38a40e773" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a464d70ec5fb4cd0b393cc467d4e9fad"
I20260812 08:03:34.231492 16103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f3091b3b2d45dc9ac7d52e2d60f5d5" candidate_uuid: "a720c9521091492f8807b8e38a40e773" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a49698551c7249ae87856561a708cf10"
I20260812 08:03:34.231590 16325 raft_consensus.cc:3060] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a464d70ec5fb4cd0b393cc467d4e9fad [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:34.231649 16103 raft_consensus.cc:3060] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:34.232684 16325 raft_consensus.cc:2468] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a464d70ec5fb4cd0b393cc467d4e9fad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720c9521091492f8807b8e38a40e773 in term 1.
I20260812 08:03:34.232774 16103 raft_consensus.cc:2468] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a720c9521091492f8807b8e38a40e773 in term 1.
I20260812 08:03:34.233110 15788 leader_election.cc:304] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a720c9521091492f8807b8e38a40e773 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a464d70ec5fb4cd0b393cc467d4e9fad, a720c9521091492f8807b8e38a40e773; no voters: 
I20260812 08:03:34.233323 16440 raft_consensus.cc:2804] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a720c9521091492f8807b8e38a40e773 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:34.233610 16440 raft_consensus.cc:697] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a720c9521091492f8807b8e38a40e773 [term 1 LEADER]: Becoming Leader. State: Replica: a720c9521091492f8807b8e38a40e773, State: Running, Role: LEADER
I20260812 08:03:34.234078 16440 consensus_queue.cc:237] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a720c9521091492f8807b8e38a40e773 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a464d70ec5fb4cd0b393cc467d4e9fad" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 34751 } } peers { permanent_uuid: "a49698551c7249ae87856561a708cf10" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 34467 } } peers { permanent_uuid: "a720c9521091492f8807b8e38a40e773" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 38967 } }
I20260812 08:03:34.238257 15658 catalog_manager.cc:5719] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a720c9521091492f8807b8e38a40e773 reported cstate change: term changed from 0 to 1, leader changed from <none> to a720c9521091492f8807b8e38a40e773 (127.15.57.129). New cstate: current_term: 1 leader_uuid: "a720c9521091492f8807b8e38a40e773" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a464d70ec5fb4cd0b393cc467d4e9fad" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a49698551c7249ae87856561a708cf10" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 34467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a720c9521091492f8807b8e38a40e773" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 38967 } health_report { overall_health: HEALTHY } } }
I20260812 08:03:34.283847 16103 raft_consensus.cc:1275] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10 [term 1 FOLLOWER]: Refusing update from remote peer a720c9521091492f8807b8e38a40e773: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:03:34.283844 16325 raft_consensus.cc:1275] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a464d70ec5fb4cd0b393cc467d4e9fad [term 1 FOLLOWER]: Refusing update from remote peer a720c9521091492f8807b8e38a40e773: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:03:34.284781 16440 consensus_queue.cc:1048] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a720c9521091492f8807b8e38a40e773 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a49698551c7249ae87856561a708cf10" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 34467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:03:34.285148 16445 consensus_queue.cc:1048] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a720c9521091492f8807b8e38a40e773 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a464d70ec5fb4cd0b393cc467d4e9fad" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:03:34.294843 16463 mvcc.cc:204] Tried to move back new op lower bound from 7317593351295983616 to 7317593351106629632. Current Snapshot: MvccSnapshot[applied={T|T < 7317593351295983616}]
I20260812 08:03:34.297439 16462 mvcc.cc:204] Tried to move back new op lower bound from 7317593351309287424 to 7317593351106629632. Current Snapshot: MvccSnapshot[applied={T|T < 7317593351295983616}]
I20260812 08:03:34.457983 16070 tablet_service.cc:1558] Processing DeleteTablet for tablet c0f3091b3b2d45dc9ac7d52e2d60f5d5 with delete_type TABLET_DATA_TOMBSTONED from {username='slave'} at 127.0.0.1:47776
I20260812 08:03:34.458774 16495 tablet_replica.cc:333] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10: stopping tablet replica
I20260812 08:03:34.459256 16495 raft_consensus.cc:2243] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 08:03:34.459625 16495 raft_consensus.cc:2272] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 08:03:34.461724 16495 ts_tablet_manager.cc:1916] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 08:03:34.473652 16495 ts_tablet_manager.cc:1929] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.34
I20260812 08:03:34.473817 16495 log.cc:1199] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10: Deleting WAL directory at /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-1/wal/wals/c0f3091b3b2d45dc9ac7d52e2d60f5d5
W20260812 08:03:34.476295 16177 generic_service.cc:139] {username='slave'} at 127.0.0.1:47776 forcing change of non-runtime-safe flag env_inject_eio_globs
I20260812 08:03:34.476423 16177 generic_service.cc:175] {username='slave'} at 127.0.0.1:47776 changed flags via RPC: env_inject_eio_globs from '*' to '/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-1/wal/tablet-meta/**'
W20260812 08:03:34.477728 16177 generic_service.cc:139] {username='slave'} at 127.0.0.1:47776 forcing change of non-runtime-safe flag env_inject_eio
I20260812 08:03:34.477814 16177 generic_service.cc:175] {username='slave'} at 127.0.0.1:47776 changed flags via RPC: env_inject_eio from '0' to '0.10'
I20260812 08:03:34.479471 16496 ts_tablet_manager.cc:933] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10: Initiating tablet copy from peer a720c9521091492f8807b8e38a40e773 (127.15.57.129:38967)
I20260812 08:03:34.480237 16496 tablet_copy_client.cc:323] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10: tablet copy: Beginning tablet copy session from remote peer at address 127.15.57.129:38967
I20260812 08:03:34.484469 15925 tablet_copy_service.cc:140] P a720c9521091492f8807b8e38a40e773: Received BeginTabletCopySession request for tablet c0f3091b3b2d45dc9ac7d52e2d60f5d5 from peer a49698551c7249ae87856561a708cf10 ({username='slave'} at 127.15.57.130:47581)
I20260812 08:03:34.484730 15925 tablet_copy_service.cc:161] P a720c9521091492f8807b8e38a40e773: Beginning new tablet copy session on tablet c0f3091b3b2d45dc9ac7d52e2d60f5d5 from peer a49698551c7249ae87856561a708cf10 at {username='slave'} at 127.15.57.130:47581: session id = a49698551c7249ae87856561a708cf10-c0f3091b3b2d45dc9ac7d52e2d60f5d5
I20260812 08:03:34.486387 15925 tablet_copy_source_session.cc:215] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a720c9521091492f8807b8e38a40e773: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 08:03:34.488237 16496 ts_tablet_manager.cc:1916] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10: Deleting tablet data with delete state TABLET_DATA_COPYING
W20260812 08:03:34.488545 16496 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260812 08:03:34.488585 16496 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260812 08:03:34.490186 16496 env_posix.cc:431] I/O error, context: INJECTED FAILURE
I20260812 08:03:34.491547 15925 tablet_copy_service.cc:342] P a720c9521091492f8807b8e38a40e773: Request end of tablet copy session a49698551c7249ae87856561a708cf10-c0f3091b3b2d45dc9ac7d52e2d60f5d5 received from {username='slave'} at 127.15.57.130:47581
I20260812 08:03:34.491694 15925 tablet_copy_service.cc:434] P a720c9521091492f8807b8e38a40e773: ending tablet copy session a49698551c7249ae87856561a708cf10-c0f3091b3b2d45dc9ac7d52e2d60f5d5 on tablet c0f3091b3b2d45dc9ac7d52e2d60f5d5 with peer a49698551c7249ae87856561a708cf10
I20260812 08:03:34.493127 16496 ts_tablet_manager.cc:1916] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
E20260812 08:03:34.494912 16496 env_posix.cc:431] I/O error, context: INJECTED FAILURE
W20260812 08:03:34.495157 16496 tablet_copy_client.cc:257] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10: tablet copy: Failed to fully clean up tablet after aborted copy: IO error: T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10: tablet copy: Failed to tombstone tablet after aborting tablet copy: Failed to write tablet metadata c0f3091b3b2d45dc9ac7d52e2d60f5d5: Failed to rename tmp file to /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-1/wal/tablet-meta/c0f3091b3b2d45dc9ac7d52e2d60f5d5: INJECTED FAILURE: Input/output error (error 5)
I20260812 08:03:34.498060 16496 ts_tablet_manager.cc:933] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10: Initiating tablet copy from peer a720c9521091492f8807b8e38a40e773 (127.15.57.129:38967)
I20260812 08:03:34.498466 16496 tablet_copy_client.cc:323] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10: tablet copy: Beginning tablet copy session from remote peer at address 127.15.57.129:38967
I20260812 08:03:34.499063 15925 tablet_copy_service.cc:140] P a720c9521091492f8807b8e38a40e773: Received BeginTabletCopySession request for tablet c0f3091b3b2d45dc9ac7d52e2d60f5d5 from peer a49698551c7249ae87856561a708cf10 ({username='slave'} at 127.15.57.130:47581)
I20260812 08:03:34.499202 15925 tablet_copy_service.cc:161] P a720c9521091492f8807b8e38a40e773: Beginning new tablet copy session on tablet c0f3091b3b2d45dc9ac7d52e2d60f5d5 from peer a49698551c7249ae87856561a708cf10 at {username='slave'} at 127.15.57.130:47581: session id = a49698551c7249ae87856561a708cf10-c0f3091b3b2d45dc9ac7d52e2d60f5d5
I20260812 08:03:34.500218 15925 tablet_copy_source_session.cc:215] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a720c9521091492f8807b8e38a40e773: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 08:03:34.500998 16496 ts_tablet_manager.cc:1916] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10: Deleting tablet data with delete state TABLET_DATA_COPYING
E20260812 08:03:34.501639 16496 env_posix.cc:431] I/O error, context: INJECTED FAILURE
E20260812 08:03:34.501807 16496 env_posix.cc:431] I/O error, context: INJECTED FAILURE
W20260812 08:03:34.501840 16496 pb_util.cc:1082] Could not delete file /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestCopyAfterFailedCopy.1786521813119228-15590-0/minicluster-data/ts-1/wal/tablet-meta/c0f3091b3b2d45dc9ac7d52e2d60f5d5.kudutmp.mCfgPm: IO error: INJECTED FAILURE: Input/output error (error 5)
I20260812 08:03:34.502615 15925 tablet_copy_service.cc:342] P a720c9521091492f8807b8e38a40e773: Request end of tablet copy session a49698551c7249ae87856561a708cf10-c0f3091b3b2d45dc9ac7d52e2d60f5d5 received from {username='slave'} at 127.15.57.130:47581
I20260812 08:03:34.502697 15925 tablet_copy_service.cc:434] P a720c9521091492f8807b8e38a40e773: ending tablet copy session a49698551c7249ae87856561a708cf10-c0f3091b3b2d45dc9ac7d52e2d60f5d5 on tablet c0f3091b3b2d45dc9ac7d52e2d60f5d5 with peer a49698551c7249ae87856561a708cf10
W20260812 08:03:34.503235 16177 generic_service.cc:139] {username='slave'} at 127.0.0.1:47776 forcing change of non-runtime-safe flag env_inject_eio
I20260812 08:03:34.503304 16177 generic_service.cc:175] {username='slave'} at 127.0.0.1:47776 changed flags via RPC: env_inject_eio from '0.10000000000000001' to '0.0'
I20260812 08:03:34.503449 16496 ts_tablet_manager.cc:1916] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 08:03:34.505389 16496 ts_tablet_manager.cc:1929] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.34
I20260812 08:03:34.506356 16496 ts_tablet_manager.cc:933] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10: Initiating tablet copy from peer a720c9521091492f8807b8e38a40e773 (127.15.57.129:38967)
I20260812 08:03:34.506656 16496 tablet_copy_client.cc:323] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10: tablet copy: Beginning tablet copy session from remote peer at address 127.15.57.129:38967
I20260812 08:03:34.507167 15925 tablet_copy_service.cc:140] P a720c9521091492f8807b8e38a40e773: Received BeginTabletCopySession request for tablet c0f3091b3b2d45dc9ac7d52e2d60f5d5 from peer a49698551c7249ae87856561a708cf10 ({username='slave'} at 127.15.57.130:47581)
I20260812 08:03:34.507299 15925 tablet_copy_service.cc:161] P a720c9521091492f8807b8e38a40e773: Beginning new tablet copy session on tablet c0f3091b3b2d45dc9ac7d52e2d60f5d5 from peer a49698551c7249ae87856561a708cf10 at {username='slave'} at 127.15.57.130:47581: session id = a49698551c7249ae87856561a708cf10-c0f3091b3b2d45dc9ac7d52e2d60f5d5
I20260812 08:03:34.508252 15925 tablet_copy_source_session.cc:215] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a720c9521091492f8807b8e38a40e773: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 08:03:34.508886 16496 ts_tablet_manager.cc:1916] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260812 08:03:34.510447 16496 ts_tablet_manager.cc:1929] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.34
I20260812 08:03:34.510658 16496 data_dirs.cc:400] Could only allocate 2 dirs of requested 3 for tablet c0f3091b3b2d45dc9ac7d52e2d60f5d5. 2 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:34.511776 16496 tablet_copy_client.cc:806] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10: tablet copy: Starting download of 0 data blocks...
I20260812 08:03:34.512004 16496 tablet_copy_client.cc:670] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10: tablet copy: Starting download of 1 WAL segments...
I20260812 08:03:34.513760 15590 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu with pid 15758
W20260812 08:03:34.534578 16496 ts_tablet_manager.cc:1011] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10: Tablet Copy: Unable to fetch data from remote peer a720c9521091492f8807b8e38a40e773 (127.15.57.129:38967): Network error: Unable to download WAL segment with seq. number 1: unable to fetch data from remote: recv got EOF from 127.15.57.129:38967 (error 108)
I20260812 08:03:34.534770 16496 tablet_replica.cc:333] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10: stopping tablet replica
I20260812 08:03:34.534849 16496 raft_consensus.cc:2243] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 08:03:34.534955 16496 raft_consensus.cc:2272] T c0f3091b3b2d45dc9ac7d52e2d60f5d5 P a49698551c7249ae87856561a708cf10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 08:03:34.534970 15590 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu with pid 15974
I20260812 08:03:34.558014 15590 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu with pid 16194
I20260812 08:03:34.592408 15590 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu with pid 15605
2026-08-12T08:03:34Z chronyd exiting
[       OK ] TabletCopyITest.TestCopyAfterFailedCopy (1498 ms)
[ RUN      ] TabletCopyITest.TestSlowCopyDoesntFail
2026-08-12T08:03:34Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:03:34Z Disabled control of system clock
I20260812 08:03:34.643559 15590 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
/tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.57.190:40745
--webserver_interface=127.15.57.190
--webserver_port=0
--builtin_ntp_servers=127.15.57.148:33485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.57.190:40745
--catalog_manager_wait_for_new_tablets_to_elect_leader=false with env {}
W20260812 08:03:34.779525 16515 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:34.779904 16515 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:34.779994 16515 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:34.783624 16515 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:03:34.783725 16515 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:34.783751 16515 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:03:34.783772 16515 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:03:34.788066 16515 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.57.148:33485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/master-0/wal
--catalog_manager_wait_for_new_tablets_to_elect_leader=false
--ipki_ca_key_size=768
--master_addresses=127.15.57.190:40745
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.57.190:40745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.57.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.16515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:34.789598 16515 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:34.791316 16515 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:34.798974 16530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:34.798974 16532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:34.798974 16534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:34.799113 16515 server_base.cc:1061] running on GCE node
I20260812 08:03:34.799978 16515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:34.801323 16515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:34.802531 16515 hybrid_clock.cc:648] HybridClock initialized: now 1786521814802515 us; error 36 us; skew 500 ppm
I20260812 08:03:34.805135 16515 webserver.cc:533] Webserver started at http://127.15.57.190:32943/ using document root <none> and password file <none>
I20260812 08:03:34.805938 16515 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:34.806037 16515 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:34.806335 16515 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:34.808425 16515 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/master-0/data/instance:
uuid: "53326e9e0846412681aafb9774a0af4c"
format_stamp: "Formatted at 2026-08-12 08:03:34 on dist-test-slave-h3ft"
I20260812 08:03:34.809032 16515 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/master-0/wal/instance:
uuid: "53326e9e0846412681aafb9774a0af4c"
format_stamp: "Formatted at 2026-08-12 08:03:34 on dist-test-slave-h3ft"
I20260812 08:03:34.813383 16515 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:03:34.816444 16542 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:34.817667 16515 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:03:34.817842 16515 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/master-0/data,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/master-0/wal
uuid: "53326e9e0846412681aafb9774a0af4c"
format_stamp: "Formatted at 2026-08-12 08:03:34 on dist-test-slave-h3ft"
I20260812 08:03:34.817982 16515 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:34.838451 16515 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:34.839597 16515 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:34.839859 16515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:34.851871 16515 rpc_server.cc:307] RPC server started. Bound to: 127.15.57.190:40745
I20260812 08:03:34.851868 16625 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.57.190:40745 every 8 connection(s)
I20260812 08:03:34.854091 16515 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/master-0/data/info.pb
I20260812 08:03:34.858316 16626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:34.861778 15590 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu as pid 16515
I20260812 08:03:34.861975 15590 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/master-0/wal/instance
I20260812 08:03:34.865836 16626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53326e9e0846412681aafb9774a0af4c: Bootstrap starting.
I20260812 08:03:34.868621 16626 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53326e9e0846412681aafb9774a0af4c: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:34.869481 16626 log.cc:826] T 00000000000000000000000000000000 P 53326e9e0846412681aafb9774a0af4c: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:34.871948 16626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53326e9e0846412681aafb9774a0af4c: No bootstrap required, opened a new log
I20260812 08:03:34.875293 16626 raft_consensus.cc:359] T 00000000000000000000000000000000 P 53326e9e0846412681aafb9774a0af4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53326e9e0846412681aafb9774a0af4c" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 40745 } }
I20260812 08:03:34.875566 16626 raft_consensus.cc:385] T 00000000000000000000000000000000 P 53326e9e0846412681aafb9774a0af4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:34.875614 16626 raft_consensus.cc:740] T 00000000000000000000000000000000 P 53326e9e0846412681aafb9774a0af4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53326e9e0846412681aafb9774a0af4c, State: Initialized, Role: FOLLOWER
I20260812 08:03:34.876037 16626 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53326e9e0846412681aafb9774a0af4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53326e9e0846412681aafb9774a0af4c" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 40745 } }
I20260812 08:03:34.876156 16626 raft_consensus.cc:399] T 00000000000000000000000000000000 P 53326e9e0846412681aafb9774a0af4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:03:34.876199 16626 raft_consensus.cc:493] T 00000000000000000000000000000000 P 53326e9e0846412681aafb9774a0af4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:03:34.876283 16626 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 53326e9e0846412681aafb9774a0af4c [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:34.877161 16626 raft_consensus.cc:515] T 00000000000000000000000000000000 P 53326e9e0846412681aafb9774a0af4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53326e9e0846412681aafb9774a0af4c" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 40745 } }
I20260812 08:03:34.877506 16626 leader_election.cc:304] T 00000000000000000000000000000000 P 53326e9e0846412681aafb9774a0af4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53326e9e0846412681aafb9774a0af4c; no voters: 
I20260812 08:03:34.877941 16626 leader_election.cc:290] T 00000000000000000000000000000000 P 53326e9e0846412681aafb9774a0af4c [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:03:34.878086 16631 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 53326e9e0846412681aafb9774a0af4c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:34.878345 16631 raft_consensus.cc:697] T 00000000000000000000000000000000 P 53326e9e0846412681aafb9774a0af4c [term 1 LEADER]: Becoming Leader. State: Replica: 53326e9e0846412681aafb9774a0af4c, State: Running, Role: LEADER
I20260812 08:03:34.878715 16631 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53326e9e0846412681aafb9774a0af4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53326e9e0846412681aafb9774a0af4c" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 40745 } }
I20260812 08:03:34.879418 16626 sys_catalog.cc:565] T 00000000000000000000000000000000 P 53326e9e0846412681aafb9774a0af4c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:03:34.880684 16632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53326e9e0846412681aafb9774a0af4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53326e9e0846412681aafb9774a0af4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53326e9e0846412681aafb9774a0af4c" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 40745 } } }
I20260812 08:03:34.880846 16632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53326e9e0846412681aafb9774a0af4c [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:34.881142 16633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53326e9e0846412681aafb9774a0af4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53326e9e0846412681aafb9774a0af4c. Latest consensus state: current_term: 1 leader_uuid: "53326e9e0846412681aafb9774a0af4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53326e9e0846412681aafb9774a0af4c" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 40745 } } }
I20260812 08:03:34.881243 16633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53326e9e0846412681aafb9774a0af4c [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:34.882004 16650 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:03:34.885459 16650 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:03:34.891935 16650 catalog_manager.cc:1383] Generated new cluster ID: 3e95a0c8edbb44cba50f3699e2354cde
I20260812 08:03:34.892019 16650 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:03:34.925203 16650 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:03:34.926717 16650 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:03:34.938963 16650 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 53326e9e0846412681aafb9774a0af4c: Generated new TSK 0
I20260812 08:03:34.940049 16650 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:03:34.944087 15590 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
/tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.57.129:0
--local_ip_for_outbound_sockets=127.15.57.129
--webserver_interface=127.15.57.129
--webserver_port=0
--tserver_master_addrs=127.15.57.190:40745
--builtin_ntp_servers=127.15.57.148:33485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--tablet_copy_download_file_inject_latency_ms=5000
--follower_unavailable_considered_failed_sec=2 with env {}
W20260812 08:03:35.081245 16668 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 08:03:35.081687 16668 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:35.081771 16668 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:35.081871 16668 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:35.085788 16668 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:35.086030 16668 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.57.129
I20260812 08:03:35.090507 16668 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.57.148:33485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=2
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.57.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.57.129
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=5000
--tserver_master_addrs=127.15.57.190:40745
--never_fsync=true
--heap_profile_path=/tmp/kudu.16668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.57.129

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:35.092226 16668 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:35.093886 16668 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:35.102947 16682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:35.102865 16679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:35.102865 16676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:35.103241 16668 server_base.cc:1061] running on GCE node
I20260812 08:03:35.103848 16668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:35.104697 16668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:35.105890 16668 hybrid_clock.cc:648] HybridClock initialized: now 1786521815105866 us; error 40 us; skew 500 ppm
I20260812 08:03:35.108505 16668 webserver.cc:533] Webserver started at http://127.15.57.129:45987/ using document root <none> and password file <none>
I20260812 08:03:35.109316 16668 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:35.109385 16668 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:35.109674 16668 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:35.111868 16668 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-0/data/instance:
uuid: "3035ef369e9c470b99c90ee728e49578"
format_stamp: "Formatted at 2026-08-12 08:03:35 on dist-test-slave-h3ft"
I20260812 08:03:35.112491 16668 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-0/wal/instance:
uuid: "3035ef369e9c470b99c90ee728e49578"
format_stamp: "Formatted at 2026-08-12 08:03:35 on dist-test-slave-h3ft"
I20260812 08:03:35.116789 16668 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:03:35.119578 16691 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:35.120919 16668 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:03:35.121074 16668 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-0/wal
uuid: "3035ef369e9c470b99c90ee728e49578"
format_stamp: "Formatted at 2026-08-12 08:03:35 on dist-test-slave-h3ft"
I20260812 08:03:35.121201 16668 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:35.157652 16668 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:35.158674 16668 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:35.158952 16668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:35.159857 16668 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:35.161252 16668 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:35.161307 16668 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:35.161387 16668 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:35.161437 16668 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:35.173761 16668 rpc_server.cc:307] RPC server started. Bound to: 127.15.57.129:34815
I20260812 08:03:35.173787 16872 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.57.129:34815 every 8 connection(s)
I20260812 08:03:35.175134 16668 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-0/data/info.pb
I20260812 08:03:35.185004 15590 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu as pid 16668
I20260812 08:03:35.185177 15590 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-0/wal/instance
I20260812 08:03:35.186024 16873 heartbeater.cc:344] Connected to a master server at 127.15.57.190:40745
I20260812 08:03:35.186410 16873 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:35.187270 16873 heartbeater.cc:507] Master 127.15.57.190:40745 requested a full tablet report, sending...
I20260812 08:03:35.188241 15590 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
/tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.57.130:0
--local_ip_for_outbound_sockets=127.15.57.130
--webserver_interface=127.15.57.130
--webserver_port=0
--tserver_master_addrs=127.15.57.190:40745
--builtin_ntp_servers=127.15.57.148:33485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--tablet_copy_download_file_inject_latency_ms=5000
--follower_unavailable_considered_failed_sec=2 with env {}
I20260812 08:03:35.189373 16565 ts_manager.cc:194] Registered new tserver with Master: 3035ef369e9c470b99c90ee728e49578 (127.15.57.129:34815)
I20260812 08:03:35.191280 16565 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.15.57.129:49547
W20260812 08:03:35.321938 16877 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 08:03:35.322270 16877 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:35.322294 16877 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:35.322342 16877 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:35.326105 16877 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:35.326247 16877 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.57.130
I20260812 08:03:35.330691 16877 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.57.148:33485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=2
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.57.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.57.130
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=5000
--tserver_master_addrs=127.15.57.190:40745
--never_fsync=true
--heap_profile_path=/tmp/kudu.16877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.57.130

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:35.332267 16877 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:35.333755 16877 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:35.342274 16887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:35.342290 16891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:35.342446 16877 server_base.cc:1061] running on GCE node
W20260812 08:03:35.342295 16888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:35.343396 16877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:35.344177 16877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:35.345356 16877 hybrid_clock.cc:648] HybridClock initialized: now 1786521815345327 us; error 46 us; skew 500 ppm
I20260812 08:03:35.347774 16877 webserver.cc:533] Webserver started at http://127.15.57.130:43709/ using document root <none> and password file <none>
I20260812 08:03:35.348527 16877 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:35.348588 16877 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:35.348877 16877 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:35.350822 16877 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-1/data/instance:
uuid: "342041438cf445928bc6287250e09edf"
format_stamp: "Formatted at 2026-08-12 08:03:35 on dist-test-slave-h3ft"
I20260812 08:03:35.351444 16877 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-1/wal/instance:
uuid: "342041438cf445928bc6287250e09edf"
format_stamp: "Formatted at 2026-08-12 08:03:35 on dist-test-slave-h3ft"
I20260812 08:03:35.355516 16877 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260812 08:03:35.358135 16901 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:35.359360 16877 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:03:35.359511 16877 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-1/wal
uuid: "342041438cf445928bc6287250e09edf"
format_stamp: "Formatted at 2026-08-12 08:03:35 on dist-test-slave-h3ft"
I20260812 08:03:35.359627 16877 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:35.374815 16877 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:35.375900 16877 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:35.376158 16877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:35.376955 16877 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:35.378320 16877 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:35.378376 16877 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:35.378463 16877 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:35.378502 16877 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:35.390702 16877 rpc_server.cc:307] RPC server started. Bound to: 127.15.57.130:37837
I20260812 08:03:35.390733 17080 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.57.130:37837 every 8 connection(s)
I20260812 08:03:35.392628 16877 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-1/data/info.pb
I20260812 08:03:35.398757 15590 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu as pid 16877
I20260812 08:03:35.398907 15590 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-1/wal/instance
I20260812 08:03:35.401960 15590 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
/tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.57.131:0
--local_ip_for_outbound_sockets=127.15.57.131
--webserver_interface=127.15.57.131
--webserver_port=0
--tserver_master_addrs=127.15.57.190:40745
--builtin_ntp_servers=127.15.57.148:33485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--tablet_copy_download_file_inject_latency_ms=5000
--follower_unavailable_considered_failed_sec=2 with env {}
I20260812 08:03:35.404058 17083 heartbeater.cc:344] Connected to a master server at 127.15.57.190:40745
I20260812 08:03:35.404417 17083 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:35.405189 17083 heartbeater.cc:507] Master 127.15.57.190:40745 requested a full tablet report, sending...
I20260812 08:03:35.406661 16565 ts_manager.cc:194] Registered new tserver with Master: 342041438cf445928bc6287250e09edf (127.15.57.130:37837)
I20260812 08:03:35.407590 16565 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.15.57.130:52363
W20260812 08:03:35.536670 17088 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 08:03:35.536978 17088 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:35.537003 17088 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:35.537055 17088 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:35.540547 17088 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:35.540740 17088 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.57.131
I20260812 08:03:35.545085 17088 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.57.148:33485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=2
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.57.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.57.131
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=5000
--tserver_master_addrs=127.15.57.190:40745
--never_fsync=true
--heap_profile_path=/tmp/kudu.17088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.57.131

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:35.546533 17088 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:35.548027 17088 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:35.556779 17097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:35.556960 17088 server_base.cc:1061] running on GCE node
W20260812 08:03:35.556972 17094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:35.556779 17100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:35.557637 17088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:35.558409 17088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:35.559628 17088 hybrid_clock.cc:648] HybridClock initialized: now 1786521815559586 us; error 39 us; skew 500 ppm
I20260812 08:03:35.562183 17088 webserver.cc:533] Webserver started at http://127.15.57.131:38271/ using document root <none> and password file <none>
I20260812 08:03:35.563076 17088 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:35.563155 17088 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:35.563478 17088 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:35.565723 17088 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-2/data/instance:
uuid: "f23fcbd60a0142fd8e0db87060c60e53"
format_stamp: "Formatted at 2026-08-12 08:03:35 on dist-test-slave-h3ft"
I20260812 08:03:35.566365 17088 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-2/wal/instance:
uuid: "f23fcbd60a0142fd8e0db87060c60e53"
format_stamp: "Formatted at 2026-08-12 08:03:35 on dist-test-slave-h3ft"
I20260812 08:03:35.570796 17088 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260812 08:03:35.573930 17112 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:35.575829 17088 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:03:35.575978 17088 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-2/wal
uuid: "f23fcbd60a0142fd8e0db87060c60e53"
format_stamp: "Formatted at 2026-08-12 08:03:35 on dist-test-slave-h3ft"
I20260812 08:03:35.576159 17088 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:35.587603 17088 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:35.588557 17088 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:35.588811 17088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:35.589603 17088 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:35.591073 17088 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:35.591136 17088 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260812 08:03:35.591223 17088 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:35.591259 17088 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:35.603757 17088 rpc_server.cc:307] RPC server started. Bound to: 127.15.57.131:42603
I20260812 08:03:35.603787 17288 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.57.131:42603 every 8 connection(s)
I20260812 08:03:35.605187 17088 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-2/data/info.pb
I20260812 08:03:35.613169 15590 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu as pid 17088
I20260812 08:03:35.613354 15590 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-2/wal/instance
I20260812 08:03:35.616894 17289 heartbeater.cc:344] Connected to a master server at 127.15.57.190:40745
I20260812 08:03:35.617254 17289 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:35.617973 17289 heartbeater.cc:507] Master 127.15.57.190:40745 requested a full tablet report, sending...
I20260812 08:03:35.619342 16565 ts_manager.cc:194] Registered new tserver with Master: f23fcbd60a0142fd8e0db87060c60e53 (127.15.57.131:42603)
I20260812 08:03:35.620031 16565 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.15.57.131:46167
I20260812 08:03:35.629415 15590 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:03:35.650738 16565 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38550:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260812 08:03:35.652408 16565 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 08:03:35.673537 17189 tablet_service.cc:1511] Processing CreateTablet for tablet 6d252e3720cc4f5093464bde02b0b920 (DEFAULT_TABLE table=test-workload [id=cbb36da0b8314c14a05f0be8bb5d80e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:35.673537 16765 tablet_service.cc:1511] Processing CreateTablet for tablet 6d252e3720cc4f5093464bde02b0b920 (DEFAULT_TABLE table=test-workload [id=cbb36da0b8314c14a05f0be8bb5d80e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:35.673535 16976 tablet_service.cc:1511] Processing CreateTablet for tablet 6d252e3720cc4f5093464bde02b0b920 (DEFAULT_TABLE table=test-workload [id=cbb36da0b8314c14a05f0be8bb5d80e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:35.674893 17189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d252e3720cc4f5093464bde02b0b920. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:35.674899 16765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d252e3720cc4f5093464bde02b0b920. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:35.674893 16976 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d252e3720cc4f5093464bde02b0b920. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:35.681721 17322 tablet_bootstrap.cc:492] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf: Bootstrap starting.
I20260812 08:03:35.682639 17324 tablet_bootstrap.cc:492] T 6d252e3720cc4f5093464bde02b0b920 P f23fcbd60a0142fd8e0db87060c60e53: Bootstrap starting.
I20260812 08:03:35.683429 17323 tablet_bootstrap.cc:492] T 6d252e3720cc4f5093464bde02b0b920 P 3035ef369e9c470b99c90ee728e49578: Bootstrap starting.
I20260812 08:03:35.685447 17324 tablet_bootstrap.cc:654] T 6d252e3720cc4f5093464bde02b0b920 P f23fcbd60a0142fd8e0db87060c60e53: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:35.685601 17322 tablet_bootstrap.cc:654] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:35.685997 17323 tablet_bootstrap.cc:654] T 6d252e3720cc4f5093464bde02b0b920 P 3035ef369e9c470b99c90ee728e49578: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:35.686703 17322 log.cc:826] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:35.686707 17324 log.cc:826] T 6d252e3720cc4f5093464bde02b0b920 P f23fcbd60a0142fd8e0db87060c60e53: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:35.686836 17323 log.cc:826] T 6d252e3720cc4f5093464bde02b0b920 P 3035ef369e9c470b99c90ee728e49578: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:35.688858 17322 tablet_bootstrap.cc:492] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf: No bootstrap required, opened a new log
I20260812 08:03:35.688858 17323 tablet_bootstrap.cc:492] T 6d252e3720cc4f5093464bde02b0b920 P 3035ef369e9c470b99c90ee728e49578: No bootstrap required, opened a new log
I20260812 08:03:35.689146 17322 ts_tablet_manager.cc:1403] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf: Time spent bootstrapping tablet: real 0.008s	user 0.006s	sys 0.000s
I20260812 08:03:35.689149 17323 ts_tablet_manager.cc:1403] T 6d252e3720cc4f5093464bde02b0b920 P 3035ef369e9c470b99c90ee728e49578: Time spent bootstrapping tablet: real 0.006s	user 0.002s	sys 0.002s
I20260812 08:03:35.690196 17324 tablet_bootstrap.cc:492] T 6d252e3720cc4f5093464bde02b0b920 P f23fcbd60a0142fd8e0db87060c60e53: No bootstrap required, opened a new log
I20260812 08:03:35.690420 17324 ts_tablet_manager.cc:1403] T 6d252e3720cc4f5093464bde02b0b920 P f23fcbd60a0142fd8e0db87060c60e53: Time spent bootstrapping tablet: real 0.008s	user 0.005s	sys 0.000s
I20260812 08:03:35.692441 17323 raft_consensus.cc:359] T 6d252e3720cc4f5093464bde02b0b920 P 3035ef369e9c470b99c90ee728e49578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3035ef369e9c470b99c90ee728e49578" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 34815 } } peers { permanent_uuid: "342041438cf445928bc6287250e09edf" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 37837 } } peers { permanent_uuid: "f23fcbd60a0142fd8e0db87060c60e53" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 42603 } }
I20260812 08:03:35.692691 17322 raft_consensus.cc:359] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3035ef369e9c470b99c90ee728e49578" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 34815 } } peers { permanent_uuid: "342041438cf445928bc6287250e09edf" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 37837 } } peers { permanent_uuid: "f23fcbd60a0142fd8e0db87060c60e53" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 42603 } }
I20260812 08:03:35.692818 17323 raft_consensus.cc:740] T 6d252e3720cc4f5093464bde02b0b920 P 3035ef369e9c470b99c90ee728e49578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3035ef369e9c470b99c90ee728e49578, State: Initialized, Role: FOLLOWER
I20260812 08:03:35.692910 17322 raft_consensus.cc:740] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 342041438cf445928bc6287250e09edf, State: Initialized, Role: FOLLOWER
I20260812 08:03:35.693249 17322 consensus_queue.cc:260] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3035ef369e9c470b99c90ee728e49578" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 34815 } } peers { permanent_uuid: "342041438cf445928bc6287250e09edf" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 37837 } } peers { permanent_uuid: "f23fcbd60a0142fd8e0db87060c60e53" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 42603 } }
I20260812 08:03:35.693253 17323 consensus_queue.cc:260] T 6d252e3720cc4f5093464bde02b0b920 P 3035ef369e9c470b99c90ee728e49578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3035ef369e9c470b99c90ee728e49578" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 34815 } } peers { permanent_uuid: "342041438cf445928bc6287250e09edf" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 37837 } } peers { permanent_uuid: "f23fcbd60a0142fd8e0db87060c60e53" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 42603 } }
I20260812 08:03:35.693923 17324 raft_consensus.cc:359] T 6d252e3720cc4f5093464bde02b0b920 P f23fcbd60a0142fd8e0db87060c60e53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3035ef369e9c470b99c90ee728e49578" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 34815 } } peers { permanent_uuid: "342041438cf445928bc6287250e09edf" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 37837 } } peers { permanent_uuid: "f23fcbd60a0142fd8e0db87060c60e53" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 42603 } }
I20260812 08:03:35.694171 17324 raft_consensus.cc:740] T 6d252e3720cc4f5093464bde02b0b920 P f23fcbd60a0142fd8e0db87060c60e53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f23fcbd60a0142fd8e0db87060c60e53, State: Initialized, Role: FOLLOWER
I20260812 08:03:35.694455 17322 ts_tablet_manager.cc:1434] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260812 08:03:35.694470 17323 ts_tablet_manager.cc:1434] T 6d252e3720cc4f5093464bde02b0b920 P 3035ef369e9c470b99c90ee728e49578: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.001s
I20260812 08:03:35.694476 17324 consensus_queue.cc:260] T 6d252e3720cc4f5093464bde02b0b920 P f23fcbd60a0142fd8e0db87060c60e53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3035ef369e9c470b99c90ee728e49578" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 34815 } } peers { permanent_uuid: "342041438cf445928bc6287250e09edf" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 37837 } } peers { permanent_uuid: "f23fcbd60a0142fd8e0db87060c60e53" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 42603 } }
I20260812 08:03:35.694917 17083 heartbeater.cc:499] Master 127.15.57.190:40745 was elected leader, sending a full tablet report...
I20260812 08:03:35.694917 16873 heartbeater.cc:499] Master 127.15.57.190:40745 was elected leader, sending a full tablet report...
I20260812 08:03:35.695472 17324 ts_tablet_manager.cc:1434] T 6d252e3720cc4f5093464bde02b0b920 P f23fcbd60a0142fd8e0db87060c60e53: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260812 08:03:35.695848 17289 heartbeater.cc:499] Master 127.15.57.190:40745 was elected leader, sending a full tablet report...
I20260812 08:03:35.729007 16800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6d252e3720cc4f5093464bde02b0b920"
dest_uuid: "3035ef369e9c470b99c90ee728e49578"
 from {username='slave'} at 127.0.0.1:34896
I20260812 08:03:35.729490 16800 raft_consensus.cc:493] T 6d252e3720cc4f5093464bde02b0b920 P 3035ef369e9c470b99c90ee728e49578 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 08:03:35.729687 16800 raft_consensus.cc:3060] T 6d252e3720cc4f5093464bde02b0b920 P 3035ef369e9c470b99c90ee728e49578 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:35.731091 16800 raft_consensus.cc:515] T 6d252e3720cc4f5093464bde02b0b920 P 3035ef369e9c470b99c90ee728e49578 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3035ef369e9c470b99c90ee728e49578" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 34815 } } peers { permanent_uuid: "342041438cf445928bc6287250e09edf" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 37837 } } peers { permanent_uuid: "f23fcbd60a0142fd8e0db87060c60e53" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 42603 } }
I20260812 08:03:35.732749 16800 leader_election.cc:290] T 6d252e3720cc4f5093464bde02b0b920 P 3035ef369e9c470b99c90ee728e49578 [CANDIDATE]: Term 1 election: Requested vote from peers 342041438cf445928bc6287250e09edf (127.15.57.130:37837), f23fcbd60a0142fd8e0db87060c60e53 (127.15.57.131:42603)
I20260812 08:03:35.738897 17012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d252e3720cc4f5093464bde02b0b920" candidate_uuid: "3035ef369e9c470b99c90ee728e49578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "342041438cf445928bc6287250e09edf"
I20260812 08:03:35.738897 17214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d252e3720cc4f5093464bde02b0b920" candidate_uuid: "3035ef369e9c470b99c90ee728e49578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f23fcbd60a0142fd8e0db87060c60e53"
I20260812 08:03:35.739254 17012 raft_consensus.cc:3060] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:35.739254 17214 raft_consensus.cc:3060] T 6d252e3720cc4f5093464bde02b0b920 P f23fcbd60a0142fd8e0db87060c60e53 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:35.740835 17012 raft_consensus.cc:2468] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3035ef369e9c470b99c90ee728e49578 in term 1.
I20260812 08:03:35.740835 17214 raft_consensus.cc:2468] T 6d252e3720cc4f5093464bde02b0b920 P f23fcbd60a0142fd8e0db87060c60e53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3035ef369e9c470b99c90ee728e49578 in term 1.
I20260812 08:03:35.741612 16696 leader_election.cc:304] T 6d252e3720cc4f5093464bde02b0b920 P 3035ef369e9c470b99c90ee728e49578 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3035ef369e9c470b99c90ee728e49578, 342041438cf445928bc6287250e09edf, f23fcbd60a0142fd8e0db87060c60e53; no voters: 
I20260812 08:03:35.742031 17331 raft_consensus.cc:2804] T 6d252e3720cc4f5093464bde02b0b920 P 3035ef369e9c470b99c90ee728e49578 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:35.742323 17331 raft_consensus.cc:697] T 6d252e3720cc4f5093464bde02b0b920 P 3035ef369e9c470b99c90ee728e49578 [term 1 LEADER]: Becoming Leader. State: Replica: 3035ef369e9c470b99c90ee728e49578, State: Running, Role: LEADER
I20260812 08:03:35.742808 17331 consensus_queue.cc:237] T 6d252e3720cc4f5093464bde02b0b920 P 3035ef369e9c470b99c90ee728e49578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3035ef369e9c470b99c90ee728e49578" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 34815 } } peers { permanent_uuid: "342041438cf445928bc6287250e09edf" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 37837 } } peers { permanent_uuid: "f23fcbd60a0142fd8e0db87060c60e53" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 42603 } }
I20260812 08:03:35.747869 16562 catalog_manager.cc:5719] T 6d252e3720cc4f5093464bde02b0b920 P 3035ef369e9c470b99c90ee728e49578 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3035ef369e9c470b99c90ee728e49578 (127.15.57.129). New cstate: current_term: 1 leader_uuid: "3035ef369e9c470b99c90ee728e49578" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3035ef369e9c470b99c90ee728e49578" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 34815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "342041438cf445928bc6287250e09edf" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 37837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f23fcbd60a0142fd8e0db87060c60e53" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 42603 } health_report { overall_health: UNKNOWN } } }
I20260812 08:03:35.755661 17012 raft_consensus.cc:1275] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf [term 1 FOLLOWER]: Refusing update from remote peer 3035ef369e9c470b99c90ee728e49578: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:03:35.755662 17214 raft_consensus.cc:1275] T 6d252e3720cc4f5093464bde02b0b920 P f23fcbd60a0142fd8e0db87060c60e53 [term 1 FOLLOWER]: Refusing update from remote peer 3035ef369e9c470b99c90ee728e49578: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:03:35.756579 17331 consensus_queue.cc:1048] T 6d252e3720cc4f5093464bde02b0b920 P 3035ef369e9c470b99c90ee728e49578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "342041438cf445928bc6287250e09edf" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 37837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:03:35.756839 17348 consensus_queue.cc:1048] T 6d252e3720cc4f5093464bde02b0b920 P 3035ef369e9c470b99c90ee728e49578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f23fcbd60a0142fd8e0db87060c60e53" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 42603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:03:35.766573 17358 mvcc.cc:204] Tried to move back new op lower bound from 7317593357329027072 to 7317593357287981056. Current Snapshot: MvccSnapshot[applied={T|T < 7317593357324222464}]
I20260812 08:03:35.771229 17357 mvcc.cc:204] Tried to move back new op lower bound from 7317593357332967424 to 7317593357287981056. Current Snapshot: MvccSnapshot[applied={T|T < 7317593357324222464}]
I20260812 08:03:35.968384 15590 tablet_copy-itest.cc:973] Tombstoning follower tablet 6d252e3720cc4f5093464bde02b0b920 on TS 342041438cf445928bc6287250e09edf
I20260812 08:03:35.969306 16976 tablet_service.cc:1558] Processing DeleteTablet for tablet 6d252e3720cc4f5093464bde02b0b920 with delete_type TABLET_DATA_TOMBSTONED from {username='slave'} at 127.0.0.1:33494
I20260812 08:03:35.970028 17388 tablet_replica.cc:333] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf: stopping tablet replica
I20260812 08:03:35.970588 17388 raft_consensus.cc:2243] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 08:03:35.970965 17388 pending_rounds.cc:70] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf: Trying to abort 1 pending ops.
I20260812 08:03:35.971179 17388 pending_rounds.cc:77] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf: Aborting op as it isn't in flight: id { term: 1 index: 105 } timestamp: 7317593358189232128 op_type: WRITE_OP write_request { tablet_id: "6d252e3720cc4f5093464bde02b0b920" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\000\305\261Rj\344E\312\314\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000" indirect_data: "<redacted>""aa79a0f4ae6c5391" } external_consistency_mode: CLIENT_PROPAGATED propagated_timestamp: 7317593358158766080 authz_token { token_data: "\010\203\324\360\323\006\"3\n\005slave\022*\n cbb36da0b8314c14a05f0be8bb5d80e8\020\001\030\001 \001(\001" signature: "<redacted>""k\353\260\337\036~\\\324Dq\257\335\234-=\265y,-\317&\264I\267\332\211\217WqI\317\240\362\\\231\301\252\251J V\346\236\265\016\016\360\026v\276\364{\354P\\\330\202\007\366A\261\263\271\351" signing_key_seq_num: 0 } } request_id { client_id: "b6d2bcb1360d49fe9e7f22f1921bd5da" seq_no: 103 first_incomplete_seq_no: 101 attempt_no: 0 }
I20260812 08:03:35.972090 17388 raft_consensus.cc:2272] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 08:03:35.972103 16695 consensus_peers.cc:597] T 6d252e3720cc4f5093464bde02b0b920 P 3035ef369e9c470b99c90ee728e49578 -> Peer 342041438cf445928bc6287250e09edf (127.15.57.130:37837): Couldn't send request to peer 342041438cf445928bc6287250e09edf. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:03:35.974941 17388 ts_tablet_manager.cc:1916] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 08:03:35.978498 17388 ts_tablet_manager.cc:1929] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.105
I20260812 08:03:35.978643 17388 log.cc:1199] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf: Deleting WAL directory at /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-1/wal/wals/6d252e3720cc4f5093464bde02b0b920
I20260812 08:03:36.850090 17390 ts_tablet_manager.cc:933] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf: Initiating tablet copy from peer 3035ef369e9c470b99c90ee728e49578 (127.15.57.129:34815)
I20260812 08:03:36.851050 17390 tablet_copy_client.cc:323] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf: tablet copy: Beginning tablet copy session from remote peer at address 127.15.57.129:34815
I20260812 08:03:36.856546 16831 tablet_copy_service.cc:140] P 3035ef369e9c470b99c90ee728e49578: Received BeginTabletCopySession request for tablet 6d252e3720cc4f5093464bde02b0b920 from peer 342041438cf445928bc6287250e09edf ({username='slave'} at 127.15.57.130:57011)
I20260812 08:03:36.856734 16831 tablet_copy_service.cc:161] P 3035ef369e9c470b99c90ee728e49578: Beginning new tablet copy session on tablet 6d252e3720cc4f5093464bde02b0b920 from peer 342041438cf445928bc6287250e09edf at {username='slave'} at 127.15.57.130:57011: session id = 342041438cf445928bc6287250e09edf-6d252e3720cc4f5093464bde02b0b920
I20260812 08:03:36.858395 16831 tablet_copy_source_session.cc:215] T 6d252e3720cc4f5093464bde02b0b920 P 3035ef369e9c470b99c90ee728e49578: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 08:03:36.861102 17390 ts_tablet_manager.cc:1916] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260812 08:03:36.864084 17390 ts_tablet_manager.cc:1929] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.105
I20260812 08:03:36.864367 17390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d252e3720cc4f5093464bde02b0b920. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:36.866123 17390 tablet_copy_client.cc:806] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf: tablet copy: Starting download of 0 data blocks...
I20260812 08:03:36.866366 17390 tablet_copy_client.cc:670] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf: tablet copy: Starting download of 1 WAL segments...
I20260812 08:03:36.872090 17391 tablet_copy_client.cc:1025] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf: tablet copy: Injecting latency into file download: 5000
I20260812 08:03:36.876871 15590 cluster_itest_util.cc:258] Not converged past 1 yet: <uninitialized op> 1.645 1.645
I20260812 08:03:36.981554 15590 cluster_itest_util.cc:258] Not converged past 1 yet: <uninitialized op> 1.645 1.645
I20260812 08:03:37.185221 15590 cluster_itest_util.cc:258] Not converged past 1 yet: <uninitialized op> 1.645 1.645
I20260812 08:03:37.488590 15590 cluster_itest_util.cc:258] Not converged past 1 yet: <uninitialized op> 1.645 1.645
I20260812 08:03:37.892956 15590 cluster_itest_util.cc:258] Not converged past 1 yet: <uninitialized op> 1.645 1.645
I20260812 08:03:38.396849 15590 cluster_itest_util.cc:258] Not converged past 1 yet: <uninitialized op> 1.645 1.645
W20260812 08:03:38.991912 16695 consensus_peers.cc:597] T 6d252e3720cc4f5093464bde02b0b920 P 3035ef369e9c470b99c90ee728e49578 -> Peer 342041438cf445928bc6287250e09edf (127.15.57.130:37837): Couldn't send request to peer 342041438cf445928bc6287250e09edf. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260812 08:03:39.001405 15590 cluster_itest_util.cc:258] Not converged past 1 yet: <uninitialized op> 1.645 1.645
I20260812 08:03:39.706064 15590 cluster_itest_util.cc:258] Not converged past 1 yet: <uninitialized op> 1.645 1.645
I20260812 08:03:40.510926 15590 cluster_itest_util.cc:258] Not converged past 1 yet: <uninitialized op> 1.645 1.645
I20260812 08:03:41.415807 15590 cluster_itest_util.cc:258] Not converged past 1 yet: <uninitialized op> 1.645 1.645
W20260812 08:03:41.574188 16695 consensus_peers.cc:597] T 6d252e3720cc4f5093464bde02b0b920 P 3035ef369e9c470b99c90ee728e49578 -> Peer 342041438cf445928bc6287250e09edf (127.15.57.130:37837): Couldn't send request to peer 342041438cf445928bc6287250e09edf. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260812 08:03:41.872905 17390 tablet_copy_client.cc:538] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 08:03:41.875378 17390 tablet_bootstrap.cc:492] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf: Bootstrap starting.
I20260812 08:03:42.209617 17390 tablet_bootstrap.cc:492] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf: Bootstrap replayed 1/1 log segments. Stats: ops{read=641 overwritten=0 applied=637 ignored=0} inserts{seen=636 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 4 replicates
I20260812 08:03:42.210188 17390 tablet_bootstrap.cc:492] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf: Bootstrap complete.
I20260812 08:03:42.210522 17390 ts_tablet_manager.cc:1403] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf: Time spent bootstrapping tablet: real 0.335s	user 0.304s	sys 0.028s
I20260812 08:03:42.211320 17390 raft_consensus.cc:359] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf [term 1 FOLLOWER]: Replica starting. Triggering 4 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3035ef369e9c470b99c90ee728e49578" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 34815 } } peers { permanent_uuid: "342041438cf445928bc6287250e09edf" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 37837 } } peers { permanent_uuid: "f23fcbd60a0142fd8e0db87060c60e53" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 42603 } }
I20260812 08:03:42.212260 17390 raft_consensus.cc:740] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 342041438cf445928bc6287250e09edf, State: Initialized, Role: FOLLOWER
I20260812 08:03:42.212436 17390 consensus_queue.cc:260] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 637, Last appended: 1.641, Last appended by leader: 641, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3035ef369e9c470b99c90ee728e49578" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 34815 } } peers { permanent_uuid: "342041438cf445928bc6287250e09edf" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 37837 } } peers { permanent_uuid: "f23fcbd60a0142fd8e0db87060c60e53" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 42603 } }
I20260812 08:03:42.212973 17390 ts_tablet_manager.cc:1434] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf: Time spent starting tablet: real 0.002s	user 0.006s	sys 0.000s
I20260812 08:03:42.214151 16831 tablet_copy_service.cc:342] P 3035ef369e9c470b99c90ee728e49578: Request end of tablet copy session 342041438cf445928bc6287250e09edf-6d252e3720cc4f5093464bde02b0b920 received from {username='slave'} at 127.15.57.130:57011
I20260812 08:03:42.214324 16831 tablet_copy_service.cc:434] P 3035ef369e9c470b99c90ee728e49578: ending tablet copy session 342041438cf445928bc6287250e09edf-6d252e3720cc4f5093464bde02b0b920 on tablet 6d252e3720cc4f5093464bde02b0b920 with peer 342041438cf445928bc6287250e09edf
I20260812 08:03:42.419739 15590 cluster_itest_util.cc:258] Not converged past 1 yet: 1.641 1.645 1.645
I20260812 08:03:42.504125 17008 raft_consensus.cc:1217] T 6d252e3720cc4f5093464bde02b0b920 P 342041438cf445928bc6287250e09edf [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.105->[1.106-1.645]   Dedup: 1.641->[1.642-1.645]
I20260812 08:03:43.500378 16976 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 08:03:43.500438 16765 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260812 08:03:43.500511 17189 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |       Address       | Status
----------------------------------+---------------------+---------
 53326e9e0846412681aafb9774a0af4c | 127.15.57.190:40745 | HEALTHY

Unusual flags for Master:
                         Flag                         |                                                                             Value                                                                             |      Tags       |         Master
------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
 catalog_manager_wait_for_new_tablets_to_elect_leader | false                                                                                                                                                         | hidden          | all 1 server(s) checked
 ipki_ca_key_size                                     | 768                                                                                                                                                           | experimental    | all 1 server(s) checked
 ipki_server_key_size                                 | 768                                                                                                                                                           | experimental    | all 1 server(s) checked
 never_fsync                                          | true                                                                                                                                                          | unsafe,advanced | all 1 server(s) checked
 openssl_security_level_override                      | 0                                                                                                                                                             | unsafe,hidden   | all 1 server(s) checked
 rpc_reuseport                                        | true                                                                                                                                                          | experimental    | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports                     | true                                                                                                                                                          | unsafe          | all 1 server(s) checked
 server_dump_info_format                              | pb                                                                                                                                                            | hidden          | all 1 server(s) checked
 server_dump_info_path                                | /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/master-0/data/info.pb | hidden          | all 1 server(s) checked
 tsk_num_rsa_bits                                     | 512                                                                                                                                                           | experimental    | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |        Value        |         Master
---------------------+---------------------+-------------------------
 builtin_ntp_servers | 127.15.57.148:33485 | all 1 server(s) checked
 time_source         | builtin             | all 1 server(s) checked

Tablet Server Summary
               UUID               |       Address       | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
 3035ef369e9c470b99c90ee728e49578 | 127.15.57.129:34815 | HEALTHY | <none>   |       1        |       0
 342041438cf445928bc6287250e09edf | 127.15.57.130:37837 | HEALTHY | <none>   |       0        |       0
 f23fcbd60a0142fd8e0db87060c60e53 | 127.15.57.131:42603 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
                    Flag                     |                                                                           Value                                                                           |      Tags       |      Tablet Server
---------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
 enable_leader_failure_detection             | false                                                                                                                                                     | unsafe          | all 3 server(s) checked
 ipki_server_key_size                        | 768                                                                                                                                                       | experimental    | all 3 server(s) checked
 local_ip_for_outbound_sockets               | 127.15.57.129                                                                                                                                             | experimental    | 127.15.57.129:34815
 local_ip_for_outbound_sockets               | 127.15.57.130                                                                                                                                             | experimental    | 127.15.57.130:37837
 local_ip_for_outbound_sockets               | 127.15.57.131                                                                                                                                             | experimental    | 127.15.57.131:42603
 never_fsync                                 | true                                                                                                                                                      | unsafe,advanced | all 3 server(s) checked
 openssl_security_level_override             | 0                                                                                                                                                         | unsafe,hidden   | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports            | true                                                                                                                                                      | unsafe          | all 3 server(s) checked
 server_dump_info_format                     | pb                                                                                                                                                        | hidden          | all 3 server(s) checked
 server_dump_info_path                       | /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-0/data/info.pb | hidden          | 127.15.57.129:34815
 server_dump_info_path                       | /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-1/data/info.pb | hidden          | 127.15.57.130:37837
 server_dump_info_path                       | /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestSlowCopyDoesntFail.1786521813119228-15590-0/minicluster-data/ts-2/data/info.pb | hidden          | 127.15.57.131:42603
 tablet_copy_download_file_inject_latency_ms | 5000                                                                                                                                                      | hidden          | all 3 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |        Value        |      Tablet Server
---------------------+---------------------+-------------------------
 builtin_ntp_servers | 127.15.57.148:33485 | all 3 server(s) checked
 time_source         | builtin             | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
     Name      | RF | Status  | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | HEALTHY | 1             | 1       | 0          | 0                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

OK
I20260812 08:03:43.531260 15590 log_verifier.cc:126] Checking tablet 6d252e3720cc4f5093464bde02b0b920
I20260812 08:03:43.691089 15590 log_verifier.cc:177] Verified matching terms for 645 ops in tablet 6d252e3720cc4f5093464bde02b0b920
I20260812 08:03:43.987133 15590 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu with pid 16668
I20260812 08:03:44.005995 15590 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu with pid 16877
I20260812 08:03:44.028009 15590 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu with pid 17088
I20260812 08:03:44.047989 15590 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu with pid 16515
2026-08-12T08:03:44Z chronyd exiting
[       OK ] TabletCopyITest.TestSlowCopyDoesntFail (9458 ms)
[ RUN      ] TabletCopyITest.TestTabletStateMetricsDuringTabletCopy
2026-08-12T08:03:44Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:03:44Z Disabled control of system clock
I20260812 08:03:44.106469 15590 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
/tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.57.190:43883
--webserver_interface=127.15.57.190
--webserver_port=0
--builtin_ntp_servers=127.15.57.148:35749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.57.190:43883 with env {}
W20260812 08:03:44.243614 17476 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:44.243952 17476 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:44.243994 17476 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:44.247799 17476 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:03:44.247900 17476 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:44.247923 17476 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:03:44.247941 17476 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:03:44.252650 17476 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.57.148:35749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.57.190:43883
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.57.190:43883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.57.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.17476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:44.254341 17476 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:44.256021 17476 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:44.263458 17491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:44.263439 17487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:44.263439 17488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:44.263891 17476 server_base.cc:1061] running on GCE node
I20260812 08:03:44.264600 17476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:44.265885 17476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:44.267100 17476 hybrid_clock.cc:648] HybridClock initialized: now 1786521824267087 us; error 41 us; skew 500 ppm
I20260812 08:03:44.269754 17476 webserver.cc:533] Webserver started at http://127.15.57.190:46151/ using document root <none> and password file <none>
I20260812 08:03:44.270612 17476 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:44.270718 17476 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:44.271100 17476 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:44.273249 17476 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/master-0/data/instance:
uuid: "d835e332ffe24bc6aa504fc64b0a9739"
format_stamp: "Formatted at 2026-08-12 08:03:44 on dist-test-slave-h3ft"
I20260812 08:03:44.273940 17476 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/master-0/wal/instance:
uuid: "d835e332ffe24bc6aa504fc64b0a9739"
format_stamp: "Formatted at 2026-08-12 08:03:44 on dist-test-slave-h3ft"
I20260812 08:03:44.278685 17476 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.006s
I20260812 08:03:44.281565 17499 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:44.282903 17476 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260812 08:03:44.283166 17476 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/master-0/data,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/master-0/wal
uuid: "d835e332ffe24bc6aa504fc64b0a9739"
format_stamp: "Formatted at 2026-08-12 08:03:44 on dist-test-slave-h3ft"
I20260812 08:03:44.283301 17476 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:44.307976 17476 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:44.309159 17476 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:44.309408 17476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:44.320880 17476 rpc_server.cc:307] RPC server started. Bound to: 127.15.57.190:43883
I20260812 08:03:44.320896 17589 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.57.190:43883 every 8 connection(s)
I20260812 08:03:44.322312 17476 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/master-0/data/info.pb
I20260812 08:03:44.325016 15590 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu as pid 17476
I20260812 08:03:44.325191 15590 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/master-0/wal/instance
I20260812 08:03:44.326290 17590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:44.334348 17590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d835e332ffe24bc6aa504fc64b0a9739: Bootstrap starting.
I20260812 08:03:44.337363 17590 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d835e332ffe24bc6aa504fc64b0a9739: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:44.338250 17590 log.cc:826] T 00000000000000000000000000000000 P d835e332ffe24bc6aa504fc64b0a9739: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:44.340521 17590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d835e332ffe24bc6aa504fc64b0a9739: No bootstrap required, opened a new log
I20260812 08:03:44.343502 17590 raft_consensus.cc:359] T 00000000000000000000000000000000 P d835e332ffe24bc6aa504fc64b0a9739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d835e332ffe24bc6aa504fc64b0a9739" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 43883 } }
I20260812 08:03:44.343734 17590 raft_consensus.cc:385] T 00000000000000000000000000000000 P d835e332ffe24bc6aa504fc64b0a9739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:44.343801 17590 raft_consensus.cc:740] T 00000000000000000000000000000000 P d835e332ffe24bc6aa504fc64b0a9739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d835e332ffe24bc6aa504fc64b0a9739, State: Initialized, Role: FOLLOWER
I20260812 08:03:44.344312 17590 consensus_queue.cc:260] T 00000000000000000000000000000000 P d835e332ffe24bc6aa504fc64b0a9739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d835e332ffe24bc6aa504fc64b0a9739" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 43883 } }
I20260812 08:03:44.344448 17590 raft_consensus.cc:399] T 00000000000000000000000000000000 P d835e332ffe24bc6aa504fc64b0a9739 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:03:44.344555 17590 raft_consensus.cc:493] T 00000000000000000000000000000000 P d835e332ffe24bc6aa504fc64b0a9739 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:03:44.344715 17590 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d835e332ffe24bc6aa504fc64b0a9739 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:44.345675 17590 raft_consensus.cc:515] T 00000000000000000000000000000000 P d835e332ffe24bc6aa504fc64b0a9739 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d835e332ffe24bc6aa504fc64b0a9739" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 43883 } }
I20260812 08:03:44.346086 17590 leader_election.cc:304] T 00000000000000000000000000000000 P d835e332ffe24bc6aa504fc64b0a9739 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d835e332ffe24bc6aa504fc64b0a9739; no voters: 
I20260812 08:03:44.346475 17590 leader_election.cc:290] T 00000000000000000000000000000000 P d835e332ffe24bc6aa504fc64b0a9739 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:03:44.346653 17602 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d835e332ffe24bc6aa504fc64b0a9739 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:44.347059 17602 raft_consensus.cc:697] T 00000000000000000000000000000000 P d835e332ffe24bc6aa504fc64b0a9739 [term 1 LEADER]: Becoming Leader. State: Replica: d835e332ffe24bc6aa504fc64b0a9739, State: Running, Role: LEADER
I20260812 08:03:44.347474 17602 consensus_queue.cc:237] T 00000000000000000000000000000000 P d835e332ffe24bc6aa504fc64b0a9739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d835e332ffe24bc6aa504fc64b0a9739" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 43883 } }
I20260812 08:03:44.348214 17590 sys_catalog.cc:565] T 00000000000000000000000000000000 P d835e332ffe24bc6aa504fc64b0a9739 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:03:44.349653 17605 sys_catalog.cc:455] T 00000000000000000000000000000000 P d835e332ffe24bc6aa504fc64b0a9739 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d835e332ffe24bc6aa504fc64b0a9739. Latest consensus state: current_term: 1 leader_uuid: "d835e332ffe24bc6aa504fc64b0a9739" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d835e332ffe24bc6aa504fc64b0a9739" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 43883 } } }
I20260812 08:03:44.349687 17604 sys_catalog.cc:455] T 00000000000000000000000000000000 P d835e332ffe24bc6aa504fc64b0a9739 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d835e332ffe24bc6aa504fc64b0a9739" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d835e332ffe24bc6aa504fc64b0a9739" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 43883 } } }
I20260812 08:03:44.349817 17605 sys_catalog.cc:458] T 00000000000000000000000000000000 P d835e332ffe24bc6aa504fc64b0a9739 [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:44.349817 17604 sys_catalog.cc:458] T 00000000000000000000000000000000 P d835e332ffe24bc6aa504fc64b0a9739 [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:44.350602 17615 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:03:44.354120 17615 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:03:44.361426 17615 catalog_manager.cc:1383] Generated new cluster ID: c4e4697221274192934a788ffca7ea39
I20260812 08:03:44.361563 17615 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:03:44.374255 17615 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:03:44.375483 17615 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:03:44.382881 17615 catalog_manager.cc:6092] T 00000000000000000000000000000000 P d835e332ffe24bc6aa504fc64b0a9739: Generated new TSK 0
I20260812 08:03:44.383739 17615 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:03:44.390744 15590 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
/tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.57.129:0
--local_ip_for_outbound_sockets=127.15.57.129
--webserver_interface=127.15.57.129
--webserver_port=0
--tserver_master_addrs=127.15.57.190:43883
--builtin_ntp_servers=127.15.57.148:35749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tablet_state_walk_min_period_ms=10 with env {}
W20260812 08:03:44.533795 17630 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:44.534096 17630 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:44.534152 17630 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:44.538024 17630 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:44.538182 17630 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.57.129
I20260812 08:03:44.542593 17630 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.57.148:35749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.57.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.57.129
--webserver_port=0
--tserver_master_addrs=127.15.57.190:43883
--tablet_state_walk_min_period_ms=10
--never_fsync=true
--heap_profile_path=/tmp/kudu.17630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.57.129

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:44.544179 17630 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:44.545729 17630 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:44.554646 17637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:44.554646 17641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:44.554684 17638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:44.554992 17630 server_base.cc:1061] running on GCE node
I20260812 08:03:44.555529 17630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:44.556205 17630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:44.557411 17630 hybrid_clock.cc:648] HybridClock initialized: now 1786521824557399 us; error 40 us; skew 500 ppm
I20260812 08:03:44.559877 17630 webserver.cc:533] Webserver started at http://127.15.57.129:44809/ using document root <none> and password file <none>
I20260812 08:03:44.560672 17630 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:44.560765 17630 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:44.561045 17630 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:44.562951 17630 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-0/data/instance:
uuid: "02397e94faa8471a91d49d676772dc08"
format_stamp: "Formatted at 2026-08-12 08:03:44 on dist-test-slave-h3ft"
I20260812 08:03:44.563565 17630 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-0/wal/instance:
uuid: "02397e94faa8471a91d49d676772dc08"
format_stamp: "Formatted at 2026-08-12 08:03:44 on dist-test-slave-h3ft"
I20260812 08:03:44.567711 17630 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260812 08:03:44.570747 17653 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:44.572394 17630 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:03:44.572636 17630 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-0/wal
uuid: "02397e94faa8471a91d49d676772dc08"
format_stamp: "Formatted at 2026-08-12 08:03:44 on dist-test-slave-h3ft"
I20260812 08:03:44.572780 17630 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:44.592131 17630 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:44.593132 17630 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:44.593390 17630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:44.594187 17630 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:44.595718 17630 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:44.595799 17630 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:44.595897 17630 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:44.595943 17630 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:44.608058 17630 rpc_server.cc:307] RPC server started. Bound to: 127.15.57.129:43645
I20260812 08:03:44.608160 17833 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.57.129:43645 every 8 connection(s)
I20260812 08:03:44.610538 17630 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-0/data/info.pb
I20260812 08:03:44.611449 15590 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu as pid 17630
I20260812 08:03:44.611601 15590 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-0/wal/instance
I20260812 08:03:44.615182 15590 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
/tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.57.130:0
--local_ip_for_outbound_sockets=127.15.57.130
--webserver_interface=127.15.57.130
--webserver_port=0
--tserver_master_addrs=127.15.57.190:43883
--builtin_ntp_servers=127.15.57.148:35749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tablet_state_walk_min_period_ms=10 with env {}
I20260812 08:03:44.624964 17835 heartbeater.cc:344] Connected to a master server at 127.15.57.190:43883
I20260812 08:03:44.625347 17835 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:44.626114 17835 heartbeater.cc:507] Master 127.15.57.190:43883 requested a full tablet report, sending...
I20260812 08:03:44.628435 17532 ts_manager.cc:194] Registered new tserver with Master: 02397e94faa8471a91d49d676772dc08 (127.15.57.129:43645)
I20260812 08:03:44.630456 17532 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.15.57.129:58963
W20260812 08:03:44.754277 17840 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:44.754588 17840 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:44.754652 17840 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:44.758926 17840 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:44.759186 17840 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.57.130
I20260812 08:03:44.764118 17840 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.57.148:35749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.57.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.57.130
--webserver_port=0
--tserver_master_addrs=127.15.57.190:43883
--tablet_state_walk_min_period_ms=10
--never_fsync=true
--heap_profile_path=/tmp/kudu.17840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.57.130

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:44.765813 17840 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:44.767520 17840 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:44.776329 17850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:44.776329 17855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:44.776361 17853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:44.776865 17840 server_base.cc:1061] running on GCE node
I20260812 08:03:44.777372 17840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:44.778147 17840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:44.779366 17840 hybrid_clock.cc:648] HybridClock initialized: now 1786521824779341 us; error 42 us; skew 500 ppm
I20260812 08:03:44.782006 17840 webserver.cc:533] Webserver started at http://127.15.57.130:39687/ using document root <none> and password file <none>
I20260812 08:03:44.782819 17840 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:44.782938 17840 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:44.783257 17840 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:44.785425 17840 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-1/data/instance:
uuid: "d26445873ef543de94ac07c030becef3"
format_stamp: "Formatted at 2026-08-12 08:03:44 on dist-test-slave-h3ft"
I20260812 08:03:44.786046 17840 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-1/wal/instance:
uuid: "d26445873ef543de94ac07c030becef3"
format_stamp: "Formatted at 2026-08-12 08:03:44 on dist-test-slave-h3ft"
I20260812 08:03:44.790681 17840 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260812 08:03:44.793780 17862 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:44.795313 17840 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:03:44.795471 17840 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-1/wal
uuid: "d26445873ef543de94ac07c030becef3"
format_stamp: "Formatted at 2026-08-12 08:03:44 on dist-test-slave-h3ft"
I20260812 08:03:44.795600 17840 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:44.818624 17840 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:44.819644 17840 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:44.819903 17840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:44.820714 17840 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:44.822075 17840 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:44.822154 17840 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:44.822257 17840 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:44.822304 17840 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:44.834798 17840 rpc_server.cc:307] RPC server started. Bound to: 127.15.57.130:39425
I20260812 08:03:44.834873 18041 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.57.130:39425 every 8 connection(s)
I20260812 08:03:44.836205 17840 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-1/data/info.pb
I20260812 08:03:44.846556 15590 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu as pid 17840
I20260812 08:03:44.846709 15590 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-1/wal/instance
I20260812 08:03:44.849828 18043 heartbeater.cc:344] Connected to a master server at 127.15.57.190:43883
I20260812 08:03:44.850366 18043 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:44.850739 15590 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
/tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.57.131:0
--local_ip_for_outbound_sockets=127.15.57.131
--webserver_interface=127.15.57.131
--webserver_port=0
--tserver_master_addrs=127.15.57.190:43883
--builtin_ntp_servers=127.15.57.148:35749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tablet_state_walk_min_period_ms=10 with env {}
I20260812 08:03:44.851225 18043 heartbeater.cc:507] Master 127.15.57.190:43883 requested a full tablet report, sending...
I20260812 08:03:44.852991 17532 ts_manager.cc:194] Registered new tserver with Master: d26445873ef543de94ac07c030becef3 (127.15.57.130:39425)
I20260812 08:03:44.854131 17532 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.15.57.130:55675
W20260812 08:03:44.985072 18048 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:44.985404 18048 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:44.985491 18048 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:44.989295 18048 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:44.989493 18048 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.57.131
I20260812 08:03:44.993927 18048 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.57.148:35749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.57.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.57.131
--webserver_port=0
--tserver_master_addrs=127.15.57.190:43883
--tablet_state_walk_min_period_ms=10
--never_fsync=true
--heap_profile_path=/tmp/kudu.18048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.57.131

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:44.995646 18048 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:44.997152 18048 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:45.006336 18056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:45.006420 18057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:45.006368 18059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:45.006491 18048 server_base.cc:1061] running on GCE node
I20260812 08:03:45.007216 18048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:45.007994 18048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:45.009208 18048 hybrid_clock.cc:648] HybridClock initialized: now 1786521825009178 us; error 46 us; skew 500 ppm
I20260812 08:03:45.011907 18048 webserver.cc:533] Webserver started at http://127.15.57.131:34499/ using document root <none> and password file <none>
I20260812 08:03:45.012802 18048 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:45.012907 18048 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:45.013209 18048 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:45.015527 18048 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-2/data/instance:
uuid: "67c9496544b84cbb8c371fcac3d75b86"
format_stamp: "Formatted at 2026-08-12 08:03:45 on dist-test-slave-h3ft"
I20260812 08:03:45.016192 18048 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-2/wal/instance:
uuid: "67c9496544b84cbb8c371fcac3d75b86"
format_stamp: "Formatted at 2026-08-12 08:03:45 on dist-test-slave-h3ft"
I20260812 08:03:45.020890 18048 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:03:45.023865 18067 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:45.025338 18048 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:03:45.025507 18048 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-2/wal
uuid: "67c9496544b84cbb8c371fcac3d75b86"
format_stamp: "Formatted at 2026-08-12 08:03:45 on dist-test-slave-h3ft"
I20260812 08:03:45.025645 18048 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:45.045527 18048 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:45.046511 18048 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:45.046769 18048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:45.047586 18048 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:45.049029 18048 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:45.049113 18048 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:45.049199 18048 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:45.049253 18048 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:45.062424 18048 rpc_server.cc:307] RPC server started. Bound to: 127.15.57.131:46299
I20260812 08:03:45.062448 18241 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.57.131:46299 every 8 connection(s)
I20260812 08:03:45.063911 18048 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-2/data/info.pb
I20260812 08:03:45.071964 15590 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu as pid 18048
I20260812 08:03:45.072142 15590 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-2/wal/instance
I20260812 08:03:45.076431 18243 heartbeater.cc:344] Connected to a master server at 127.15.57.190:43883
I20260812 08:03:45.076820 18243 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:45.077595 18243 heartbeater.cc:507] Master 127.15.57.190:43883 requested a full tablet report, sending...
I20260812 08:03:45.079224 17532 ts_manager.cc:194] Registered new tserver with Master: 67c9496544b84cbb8c371fcac3d75b86 (127.15.57.131:46299)
I20260812 08:03:45.079924 17532 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.15.57.131:55809
I20260812 08:03:45.088668 15590 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 08:03:45.115213 17532 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51024:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260812 08:03:45.116982 17532 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 08:03:45.139128 17936 tablet_service.cc:1511] Processing CreateTablet for tablet 33166262662c4adfb8465359eb4b2ed4 (DEFAULT_TABLE table=test-workload [id=a73355f8e0bf4eb1b84a399ccd075240]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:45.139114 17725 tablet_service.cc:1511] Processing CreateTablet for tablet 33166262662c4adfb8465359eb4b2ed4 (DEFAULT_TABLE table=test-workload [id=a73355f8e0bf4eb1b84a399ccd075240]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:45.139114 18137 tablet_service.cc:1511] Processing CreateTablet for tablet 33166262662c4adfb8465359eb4b2ed4 (DEFAULT_TABLE table=test-workload [id=a73355f8e0bf4eb1b84a399ccd075240]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:45.140440 17936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33166262662c4adfb8465359eb4b2ed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:45.140440 17725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33166262662c4adfb8465359eb4b2ed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:45.140440 18137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33166262662c4adfb8465359eb4b2ed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:45.147500 18293 tablet_bootstrap.cc:492] T 33166262662c4adfb8465359eb4b2ed4 P d26445873ef543de94ac07c030becef3: Bootstrap starting.
I20260812 08:03:45.148702 18291 tablet_bootstrap.cc:492] T 33166262662c4adfb8465359eb4b2ed4 P 67c9496544b84cbb8c371fcac3d75b86: Bootstrap starting.
I20260812 08:03:45.148712 18290 tablet_bootstrap.cc:492] T 33166262662c4adfb8465359eb4b2ed4 P 02397e94faa8471a91d49d676772dc08: Bootstrap starting.
I20260812 08:03:45.150552 18293 tablet_bootstrap.cc:654] T 33166262662c4adfb8465359eb4b2ed4 P d26445873ef543de94ac07c030becef3: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:45.151172 18290 tablet_bootstrap.cc:654] T 33166262662c4adfb8465359eb4b2ed4 P 02397e94faa8471a91d49d676772dc08: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:45.151194 18291 tablet_bootstrap.cc:654] T 33166262662c4adfb8465359eb4b2ed4 P 67c9496544b84cbb8c371fcac3d75b86: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:45.151789 18293 log.cc:826] T 33166262662c4adfb8465359eb4b2ed4 P d26445873ef543de94ac07c030becef3: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:45.152109 18290 log.cc:826] T 33166262662c4adfb8465359eb4b2ed4 P 02397e94faa8471a91d49d676772dc08: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:45.152201 18291 log.cc:826] T 33166262662c4adfb8465359eb4b2ed4 P 67c9496544b84cbb8c371fcac3d75b86: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:45.154251 18290 tablet_bootstrap.cc:492] T 33166262662c4adfb8465359eb4b2ed4 P 02397e94faa8471a91d49d676772dc08: No bootstrap required, opened a new log
I20260812 08:03:45.154356 18293 tablet_bootstrap.cc:492] T 33166262662c4adfb8465359eb4b2ed4 P d26445873ef543de94ac07c030becef3: No bootstrap required, opened a new log
I20260812 08:03:45.154434 18291 tablet_bootstrap.cc:492] T 33166262662c4adfb8465359eb4b2ed4 P 67c9496544b84cbb8c371fcac3d75b86: No bootstrap required, opened a new log
I20260812 08:03:45.154559 18290 ts_tablet_manager.cc:1403] T 33166262662c4adfb8465359eb4b2ed4 P 02397e94faa8471a91d49d676772dc08: Time spent bootstrapping tablet: real 0.006s	user 0.000s	sys 0.004s
I20260812 08:03:45.154583 18293 ts_tablet_manager.cc:1403] T 33166262662c4adfb8465359eb4b2ed4 P d26445873ef543de94ac07c030becef3: Time spent bootstrapping tablet: real 0.007s	user 0.005s	sys 0.000s
I20260812 08:03:45.154624 18291 ts_tablet_manager.cc:1403] T 33166262662c4adfb8465359eb4b2ed4 P 67c9496544b84cbb8c371fcac3d75b86: Time spent bootstrapping tablet: real 0.006s	user 0.001s	sys 0.004s
I20260812 08:03:45.157589 18290 raft_consensus.cc:359] T 33166262662c4adfb8465359eb4b2ed4 P 02397e94faa8471a91d49d676772dc08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26445873ef543de94ac07c030becef3" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 39425 } } peers { permanent_uuid: "02397e94faa8471a91d49d676772dc08" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 43645 } } peers { permanent_uuid: "67c9496544b84cbb8c371fcac3d75b86" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 46299 } }
I20260812 08:03:45.157872 18290 raft_consensus.cc:385] T 33166262662c4adfb8465359eb4b2ed4 P 02397e94faa8471a91d49d676772dc08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:45.157949 18290 raft_consensus.cc:740] T 33166262662c4adfb8465359eb4b2ed4 P 02397e94faa8471a91d49d676772dc08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02397e94faa8471a91d49d676772dc08, State: Initialized, Role: FOLLOWER
I20260812 08:03:45.158345 18291 raft_consensus.cc:359] T 33166262662c4adfb8465359eb4b2ed4 P 67c9496544b84cbb8c371fcac3d75b86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26445873ef543de94ac07c030becef3" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 39425 } } peers { permanent_uuid: "02397e94faa8471a91d49d676772dc08" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 43645 } } peers { permanent_uuid: "67c9496544b84cbb8c371fcac3d75b86" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 46299 } }
I20260812 08:03:45.158341 18293 raft_consensus.cc:359] T 33166262662c4adfb8465359eb4b2ed4 P d26445873ef543de94ac07c030becef3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26445873ef543de94ac07c030becef3" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 39425 } } peers { permanent_uuid: "02397e94faa8471a91d49d676772dc08" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 43645 } } peers { permanent_uuid: "67c9496544b84cbb8c371fcac3d75b86" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 46299 } }
I20260812 08:03:45.158591 18291 raft_consensus.cc:385] T 33166262662c4adfb8465359eb4b2ed4 P 67c9496544b84cbb8c371fcac3d75b86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:45.158591 18293 raft_consensus.cc:385] T 33166262662c4adfb8465359eb4b2ed4 P d26445873ef543de94ac07c030becef3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:45.158670 18293 raft_consensus.cc:740] T 33166262662c4adfb8465359eb4b2ed4 P d26445873ef543de94ac07c030becef3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d26445873ef543de94ac07c030becef3, State: Initialized, Role: FOLLOWER
I20260812 08:03:45.158670 18291 raft_consensus.cc:740] T 33166262662c4adfb8465359eb4b2ed4 P 67c9496544b84cbb8c371fcac3d75b86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67c9496544b84cbb8c371fcac3d75b86, State: Initialized, Role: FOLLOWER
I20260812 08:03:45.158577 18290 consensus_queue.cc:260] T 33166262662c4adfb8465359eb4b2ed4 P 02397e94faa8471a91d49d676772dc08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26445873ef543de94ac07c030becef3" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 39425 } } peers { permanent_uuid: "02397e94faa8471a91d49d676772dc08" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 43645 } } peers { permanent_uuid: "67c9496544b84cbb8c371fcac3d75b86" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 46299 } }
I20260812 08:03:45.159204 18293 consensus_queue.cc:260] T 33166262662c4adfb8465359eb4b2ed4 P d26445873ef543de94ac07c030becef3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26445873ef543de94ac07c030becef3" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 39425 } } peers { permanent_uuid: "02397e94faa8471a91d49d676772dc08" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 43645 } } peers { permanent_uuid: "67c9496544b84cbb8c371fcac3d75b86" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 46299 } }
I20260812 08:03:45.159204 18291 consensus_queue.cc:260] T 33166262662c4adfb8465359eb4b2ed4 P 67c9496544b84cbb8c371fcac3d75b86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26445873ef543de94ac07c030becef3" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 39425 } } peers { permanent_uuid: "02397e94faa8471a91d49d676772dc08" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 43645 } } peers { permanent_uuid: "67c9496544b84cbb8c371fcac3d75b86" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 46299 } }
I20260812 08:03:45.159871 18290 ts_tablet_manager.cc:1434] T 33166262662c4adfb8465359eb4b2ed4 P 02397e94faa8471a91d49d676772dc08: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.003s
I20260812 08:03:45.160171 18291 ts_tablet_manager.cc:1434] T 33166262662c4adfb8465359eb4b2ed4 P 67c9496544b84cbb8c371fcac3d75b86: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260812 08:03:45.160331 17835 heartbeater.cc:499] Master 127.15.57.190:43883 was elected leader, sending a full tablet report...
I20260812 08:03:45.160554 18243 heartbeater.cc:499] Master 127.15.57.190:43883 was elected leader, sending a full tablet report...
I20260812 08:03:45.161859 18293 ts_tablet_manager.cc:1434] T 33166262662c4adfb8465359eb4b2ed4 P d26445873ef543de94ac07c030becef3: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260812 08:03:45.162271 18043 heartbeater.cc:499] Master 127.15.57.190:43883 was elected leader, sending a full tablet report...
I20260812 08:03:45.289251 18298 raft_consensus.cc:493] T 33166262662c4adfb8465359eb4b2ed4 P 02397e94faa8471a91d49d676772dc08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:45.289585 18298 raft_consensus.cc:515] T 33166262662c4adfb8465359eb4b2ed4 P 02397e94faa8471a91d49d676772dc08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26445873ef543de94ac07c030becef3" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 39425 } } peers { permanent_uuid: "02397e94faa8471a91d49d676772dc08" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 43645 } } peers { permanent_uuid: "67c9496544b84cbb8c371fcac3d75b86" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 46299 } }
I20260812 08:03:45.291275 18298 leader_election.cc:290] T 33166262662c4adfb8465359eb4b2ed4 P 02397e94faa8471a91d49d676772dc08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d26445873ef543de94ac07c030becef3 (127.15.57.130:39425), 67c9496544b84cbb8c371fcac3d75b86 (127.15.57.131:46299)
I20260812 08:03:45.296942 17963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33166262662c4adfb8465359eb4b2ed4" candidate_uuid: "02397e94faa8471a91d49d676772dc08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d26445873ef543de94ac07c030becef3" is_pre_election: true
I20260812 08:03:45.297475 17963 raft_consensus.cc:2468] T 33166262662c4adfb8465359eb4b2ed4 P d26445873ef543de94ac07c030becef3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02397e94faa8471a91d49d676772dc08 in term 0.
I20260812 08:03:45.297495 18170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33166262662c4adfb8465359eb4b2ed4" candidate_uuid: "02397e94faa8471a91d49d676772dc08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67c9496544b84cbb8c371fcac3d75b86" is_pre_election: true
I20260812 08:03:45.297796 18170 raft_consensus.cc:2468] T 33166262662c4adfb8465359eb4b2ed4 P 67c9496544b84cbb8c371fcac3d75b86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02397e94faa8471a91d49d676772dc08 in term 0.
I20260812 08:03:45.298322 17659 leader_election.cc:304] T 33166262662c4adfb8465359eb4b2ed4 P 02397e94faa8471a91d49d676772dc08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02397e94faa8471a91d49d676772dc08, d26445873ef543de94ac07c030becef3; no voters: 
I20260812 08:03:45.298738 18298 raft_consensus.cc:2804] T 33166262662c4adfb8465359eb4b2ed4 P 02397e94faa8471a91d49d676772dc08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:03:45.298902 18298 raft_consensus.cc:493] T 33166262662c4adfb8465359eb4b2ed4 P 02397e94faa8471a91d49d676772dc08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:03:45.299003 18298 raft_consensus.cc:3060] T 33166262662c4adfb8465359eb4b2ed4 P 02397e94faa8471a91d49d676772dc08 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:45.300302 18298 raft_consensus.cc:515] T 33166262662c4adfb8465359eb4b2ed4 P 02397e94faa8471a91d49d676772dc08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26445873ef543de94ac07c030becef3" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 39425 } } peers { permanent_uuid: "02397e94faa8471a91d49d676772dc08" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 43645 } } peers { permanent_uuid: "67c9496544b84cbb8c371fcac3d75b86" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 46299 } }
I20260812 08:03:45.300944 18298 leader_election.cc:290] T 33166262662c4adfb8465359eb4b2ed4 P 02397e94faa8471a91d49d676772dc08 [CANDIDATE]: Term 1 election: Requested vote from peers d26445873ef543de94ac07c030becef3 (127.15.57.130:39425), 67c9496544b84cbb8c371fcac3d75b86 (127.15.57.131:46299)
I20260812 08:03:45.301545 18170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33166262662c4adfb8465359eb4b2ed4" candidate_uuid: "02397e94faa8471a91d49d676772dc08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67c9496544b84cbb8c371fcac3d75b86"
I20260812 08:03:45.301544 17963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33166262662c4adfb8465359eb4b2ed4" candidate_uuid: "02397e94faa8471a91d49d676772dc08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d26445873ef543de94ac07c030becef3"
I20260812 08:03:45.301730 17963 raft_consensus.cc:3060] T 33166262662c4adfb8465359eb4b2ed4 P d26445873ef543de94ac07c030becef3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:45.301730 18170 raft_consensus.cc:3060] T 33166262662c4adfb8465359eb4b2ed4 P 67c9496544b84cbb8c371fcac3d75b86 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:45.303503 18170 raft_consensus.cc:2468] T 33166262662c4adfb8465359eb4b2ed4 P 67c9496544b84cbb8c371fcac3d75b86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02397e94faa8471a91d49d676772dc08 in term 1.
I20260812 08:03:45.304065 17655 leader_election.cc:304] T 33166262662c4adfb8465359eb4b2ed4 P 02397e94faa8471a91d49d676772dc08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02397e94faa8471a91d49d676772dc08, 67c9496544b84cbb8c371fcac3d75b86; no voters: 
I20260812 08:03:45.304114 17963 raft_consensus.cc:2468] T 33166262662c4adfb8465359eb4b2ed4 P d26445873ef543de94ac07c030becef3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02397e94faa8471a91d49d676772dc08 in term 1.
I20260812 08:03:45.304345 18298 raft_consensus.cc:2804] T 33166262662c4adfb8465359eb4b2ed4 P 02397e94faa8471a91d49d676772dc08 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:45.304709 18298 raft_consensus.cc:697] T 33166262662c4adfb8465359eb4b2ed4 P 02397e94faa8471a91d49d676772dc08 [term 1 LEADER]: Becoming Leader. State: Replica: 02397e94faa8471a91d49d676772dc08, State: Running, Role: LEADER
I20260812 08:03:45.305140 18298 consensus_queue.cc:237] T 33166262662c4adfb8465359eb4b2ed4 P 02397e94faa8471a91d49d676772dc08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26445873ef543de94ac07c030becef3" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 39425 } } peers { permanent_uuid: "02397e94faa8471a91d49d676772dc08" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 43645 } } peers { permanent_uuid: "67c9496544b84cbb8c371fcac3d75b86" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 46299 } }
I20260812 08:03:45.309902 17531 catalog_manager.cc:5719] T 33166262662c4adfb8465359eb4b2ed4 P 02397e94faa8471a91d49d676772dc08 reported cstate change: term changed from 0 to 1, leader changed from <none> to 02397e94faa8471a91d49d676772dc08 (127.15.57.129). New cstate: current_term: 1 leader_uuid: "02397e94faa8471a91d49d676772dc08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26445873ef543de94ac07c030becef3" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 39425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02397e94faa8471a91d49d676772dc08" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 43645 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "67c9496544b84cbb8c371fcac3d75b86" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 46299 } health_report { overall_health: UNKNOWN } } }
W20260812 08:03:45.315194 18244 tablet.cc:2406] T 33166262662c4adfb8465359eb4b2ed4 P 67c9496544b84cbb8c371fcac3d75b86: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:03:45.337939 18044 tablet.cc:2406] T 33166262662c4adfb8465359eb4b2ed4 P d26445873ef543de94ac07c030becef3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:03:45.362664 17837 tablet.cc:2406] T 33166262662c4adfb8465359eb4b2ed4 P 02397e94faa8471a91d49d676772dc08: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:03:45.363608 17963 raft_consensus.cc:1275] T 33166262662c4adfb8465359eb4b2ed4 P d26445873ef543de94ac07c030becef3 [term 1 FOLLOWER]: Refusing update from remote peer 02397e94faa8471a91d49d676772dc08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:03:45.363610 18170 raft_consensus.cc:1275] T 33166262662c4adfb8465359eb4b2ed4 P 67c9496544b84cbb8c371fcac3d75b86 [term 1 FOLLOWER]: Refusing update from remote peer 02397e94faa8471a91d49d676772dc08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:03:45.364427 18298 consensus_queue.cc:1048] T 33166262662c4adfb8465359eb4b2ed4 P 02397e94faa8471a91d49d676772dc08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d26445873ef543de94ac07c030becef3" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 39425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:03:45.364643 18309 consensus_queue.cc:1048] T 33166262662c4adfb8465359eb4b2ed4 P 02397e94faa8471a91d49d676772dc08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67c9496544b84cbb8c371fcac3d75b86" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 46299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:03:45.376376 18320 mvcc.cc:204] Tried to move back new op lower bound from 7317593396689657856 to 7317593396454006784. Current Snapshot: MvccSnapshot[applied={T|T < 7317593396678803456}]
I20260812 08:03:45.383200 18321 mvcc.cc:204] Tried to move back new op lower bound from 7317593396689657856 to 7317593396454006784. Current Snapshot: MvccSnapshot[applied={T|T < 7317593396678803456}]
I20260812 08:03:45.504736 17936 tablet_service.cc:1558] Processing DeleteTablet for tablet 33166262662c4adfb8465359eb4b2ed4 with delete_type TABLET_DATA_TOMBSTONED from {username='slave'} at 127.0.0.1:41464
I20260812 08:03:45.505422 18358 tablet_replica.cc:333] T 33166262662c4adfb8465359eb4b2ed4 P d26445873ef543de94ac07c030becef3: stopping tablet replica
I20260812 08:03:45.505771 18358 raft_consensus.cc:2243] T 33166262662c4adfb8465359eb4b2ed4 P d26445873ef543de94ac07c030becef3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 08:03:45.506091 18358 raft_consensus.cc:2272] T 33166262662c4adfb8465359eb4b2ed4 P d26445873ef543de94ac07c030becef3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 08:03:45.507910 18358 ts_tablet_manager.cc:1916] T 33166262662c4adfb8465359eb4b2ed4 P d26445873ef543de94ac07c030becef3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 08:03:45.510198 18358 ts_tablet_manager.cc:1929] T 33166262662c4adfb8465359eb4b2ed4 P d26445873ef543de94ac07c030becef3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.25
I20260812 08:03:45.510310 18358 log.cc:1199] T 33166262662c4adfb8465359eb4b2ed4 P d26445873ef543de94ac07c030becef3: Deleting WAL directory at /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.TabletCopyITest.TestTabletStateMetricsDuringTabletCopy.1786521813119228-15590-0/minicluster-data/ts-1/wal/wals/33166262662c4adfb8465359eb4b2ed4
W20260812 08:03:46.091832 17659 consensus_peers.cc:597] T 33166262662c4adfb8465359eb4b2ed4 P 02397e94faa8471a91d49d676772dc08 -> Peer d26445873ef543de94ac07c030becef3 (127.15.57.130:39425): Couldn't send request to peer d26445873ef543de94ac07c030becef3. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:03:46.652794 18364 ts_tablet_manager.cc:933] T 33166262662c4adfb8465359eb4b2ed4 P d26445873ef543de94ac07c030becef3: Initiating tablet copy from peer 02397e94faa8471a91d49d676772dc08 (127.15.57.129:43645)
I20260812 08:03:46.653664 18364 tablet_copy_client.cc:323] T 33166262662c4adfb8465359eb4b2ed4 P d26445873ef543de94ac07c030becef3: tablet copy: Beginning tablet copy session from remote peer at address 127.15.57.129:43645
I20260812 08:03:46.658571 17783 tablet_copy_service.cc:140] P 02397e94faa8471a91d49d676772dc08: Received BeginTabletCopySession request for tablet 33166262662c4adfb8465359eb4b2ed4 from peer d26445873ef543de94ac07c030becef3 ({username='slave'} at 127.15.57.130:51407)
I20260812 08:03:46.658802 17783 tablet_copy_service.cc:161] P 02397e94faa8471a91d49d676772dc08: Beginning new tablet copy session on tablet 33166262662c4adfb8465359eb4b2ed4 from peer d26445873ef543de94ac07c030becef3 at {username='slave'} at 127.15.57.130:51407: session id = d26445873ef543de94ac07c030becef3-33166262662c4adfb8465359eb4b2ed4
I20260812 08:03:46.660255 17783 tablet_copy_source_session.cc:215] T 33166262662c4adfb8465359eb4b2ed4 P 02397e94faa8471a91d49d676772dc08: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 08:03:46.662048 18364 ts_tablet_manager.cc:1916] T 33166262662c4adfb8465359eb4b2ed4 P d26445873ef543de94ac07c030becef3: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260812 08:03:46.664263 18364 ts_tablet_manager.cc:1929] T 33166262662c4adfb8465359eb4b2ed4 P d26445873ef543de94ac07c030becef3: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.25
I20260812 08:03:46.664528 18364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33166262662c4adfb8465359eb4b2ed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:46.665783 18364 tablet_copy_client.cc:806] T 33166262662c4adfb8465359eb4b2ed4 P d26445873ef543de94ac07c030becef3: tablet copy: Starting download of 0 data blocks...
I20260812 08:03:46.665997 18364 tablet_copy_client.cc:670] T 33166262662c4adfb8465359eb4b2ed4 P d26445873ef543de94ac07c030becef3: tablet copy: Starting download of 1 WAL segments...
I20260812 08:03:46.668848 18364 tablet_copy_client.cc:538] T 33166262662c4adfb8465359eb4b2ed4 P d26445873ef543de94ac07c030becef3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 08:03:46.671141 18364 tablet_bootstrap.cc:492] T 33166262662c4adfb8465359eb4b2ed4 P d26445873ef543de94ac07c030becef3: Bootstrap starting.
I20260812 08:03:46.681008 15590 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu with pid 17630
I20260812 08:03:46.704330 15590 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu with pid 17840
I20260812 08:03:46.727229 15590 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu with pid 18048
I20260812 08:03:46.770057 15590 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu with pid 17476
2026-08-12T08:03:46Z chronyd exiting
[       OK ] TabletCopyITest.TestTabletStateMetricsDuringTabletCopy (2721 ms)
[----------] 3 tests from TabletCopyITest (13679 ms total)

[----------] 1 test from FailureCause/TabletCopyFailureITest
[ RUN      ] FailureCause/TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote/1
2026-08-12T08:03:46Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:03:46Z Disabled control of system clock
I20260812 08:03:46.826426 15590 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
/tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.57.190:45861
--webserver_interface=127.15.57.190
--webserver_port=0
--builtin_ntp_servers=127.15.57.148:40311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.57.190:45861
--catalog_manager_evict_excess_replicas=false
--master_add_server_when_underreplicated=false with env {}
W20260812 08:03:46.970201 18377 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:46.970610 18377 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:46.970702 18377 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:46.974721 18377 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:03:46.974844 18377 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:46.974895 18377 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:03:46.974954 18377 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:03:46.979624 18377 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.57.148:40311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/wal
--catalog_manager_evict_excess_replicas=false
--master_add_server_when_underreplicated=false
--ipki_ca_key_size=768
--master_addresses=127.15.57.190:45861
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.57.190:45861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.57.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.18377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:46.981257 18377 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:46.982803 18377 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:46.990170 18395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:46.990272 18390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:46.990185 18386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:46.990471 18377 server_base.cc:1061] running on GCE node
I20260812 08:03:46.991286 18377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:46.992623 18377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:46.993832 18377 hybrid_clock.cc:648] HybridClock initialized: now 1786521826993817 us; error 29 us; skew 500 ppm
I20260812 08:03:46.996250 18377 webserver.cc:533] Webserver started at http://127.15.57.190:43187/ using document root <none> and password file <none>
I20260812 08:03:46.997071 18377 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:46.997174 18377 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:46.997476 18377 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:46.999635 18377 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/data/instance:
uuid: "f64aa79d48dc45b7ab0ee33054daa8de"
format_stamp: "Formatted at 2026-08-12 08:03:46 on dist-test-slave-h3ft"
I20260812 08:03:47.000346 18377 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/wal/instance:
uuid: "f64aa79d48dc45b7ab0ee33054daa8de"
format_stamp: "Formatted at 2026-08-12 08:03:46 on dist-test-slave-h3ft"
I20260812 08:03:47.005149 18377 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:03:47.008260 18402 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:47.009660 18377 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:03:47.009848 18377 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/data,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/wal
uuid: "f64aa79d48dc45b7ab0ee33054daa8de"
format_stamp: "Formatted at 2026-08-12 08:03:46 on dist-test-slave-h3ft"
I20260812 08:03:47.009999 18377 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:47.042650 18377 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:47.043738 18377 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:47.043999 18377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:47.053476 18484 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.57.190:45861 every 8 connection(s)
I20260812 08:03:47.053488 18377 rpc_server.cc:307] RPC server started. Bound to: 127.15.57.190:45861
I20260812 08:03:47.054874 18377 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/data/info.pb
I20260812 08:03:47.055910 15590 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu as pid 18377
I20260812 08:03:47.056103 15590 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/wal/instance
I20260812 08:03:47.059587 18485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:47.065382 18485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de: Bootstrap starting.
I20260812 08:03:47.068284 18485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:47.069136 18485 log.cc:826] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:47.071625 18485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de: No bootstrap required, opened a new log
I20260812 08:03:47.074802 18485 raft_consensus.cc:359] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f64aa79d48dc45b7ab0ee33054daa8de" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 45861 } }
I20260812 08:03:47.075196 18485 raft_consensus.cc:385] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:47.075295 18485 raft_consensus.cc:740] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f64aa79d48dc45b7ab0ee33054daa8de, State: Initialized, Role: FOLLOWER
I20260812 08:03:47.075879 18485 consensus_queue.cc:260] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f64aa79d48dc45b7ab0ee33054daa8de" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 45861 } }
I20260812 08:03:47.076074 18485 raft_consensus.cc:399] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:03:47.076174 18485 raft_consensus.cc:493] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:03:47.076349 18485 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:47.077567 18485 raft_consensus.cc:515] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f64aa79d48dc45b7ab0ee33054daa8de" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 45861 } }
I20260812 08:03:47.078047 18485 leader_election.cc:304] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f64aa79d48dc45b7ab0ee33054daa8de; no voters: 
I20260812 08:03:47.078507 18485 leader_election.cc:290] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:03:47.078681 18495 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:47.078997 18495 raft_consensus.cc:697] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [term 1 LEADER]: Becoming Leader. State: Replica: f64aa79d48dc45b7ab0ee33054daa8de, State: Running, Role: LEADER
I20260812 08:03:47.079401 18495 consensus_queue.cc:237] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f64aa79d48dc45b7ab0ee33054daa8de" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 45861 } }
I20260812 08:03:47.080121 18485 sys_catalog.cc:565] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:03:47.081423 18497 sys_catalog.cc:455] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [sys.catalog]: SysCatalogTable state changed. Reason: New leader f64aa79d48dc45b7ab0ee33054daa8de. Latest consensus state: current_term: 1 leader_uuid: "f64aa79d48dc45b7ab0ee33054daa8de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f64aa79d48dc45b7ab0ee33054daa8de" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 45861 } } }
I20260812 08:03:47.081588 18497 sys_catalog.cc:458] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:47.082872 18496 sys_catalog.cc:455] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f64aa79d48dc45b7ab0ee33054daa8de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f64aa79d48dc45b7ab0ee33054daa8de" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 45861 } } }
I20260812 08:03:47.083096 18496 sys_catalog.cc:458] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [sys.catalog]: This master's current role is: LEADER
W20260812 08:03:47.086629 18516 catalog_manager.cc:1594] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 08:03:47.086728 18516 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 08:03:47.086843 18507 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:03:47.087822 18507 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:03:47.095316 18507 catalog_manager.cc:1383] Generated new cluster ID: 30b1224f99d7457a925b4463860fa739
I20260812 08:03:47.095453 18507 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:03:47.107445 18507 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:03:47.108579 18507 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:03:47.115610 18507 catalog_manager.cc:6092] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de: Generated new TSK 0
I20260812 08:03:47.116479 18507 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:03:47.122833 15590 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
/tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.57.129:0
--local_ip_for_outbound_sockets=127.15.57.129
--webserver_interface=127.15.57.129
--webserver_port=0
--tserver_master_addrs=127.15.57.190:45861
--builtin_ntp_servers=127.15.57.148:40311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tablet_copy_fault_crash_on_fetch_all=1.0 with env {}
W20260812 08:03:47.257928 18519 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:47.258275 18519 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:47.258322 18519 flags.cc:432] Enabled unsafe flag: --tablet_copy_fault_crash_on_fetch_all=1
W20260812 08:03:47.258383 18519 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:47.262422 18519 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:47.262575 18519 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.57.129
I20260812 08:03:47.266987 18519 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.57.148:40311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.57.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.57.129
--webserver_port=0
--tablet_copy_fault_crash_on_fetch_all=1
--tserver_master_addrs=127.15.57.190:45861
--never_fsync=true
--heap_profile_path=/tmp/kudu.18519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.57.129

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:47.268679 18519 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:47.270215 18519 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:47.278781 18535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:47.278903 18529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:47.278826 18532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:47.279225 18519 server_base.cc:1061] running on GCE node
I20260812 08:03:47.279667 18519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:47.280366 18519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:47.281577 18519 hybrid_clock.cc:648] HybridClock initialized: now 1786521827281555 us; error 36 us; skew 500 ppm
I20260812 08:03:47.283881 18519 webserver.cc:533] Webserver started at http://127.15.57.129:38195/ using document root <none> and password file <none>
I20260812 08:03:47.284617 18519 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:47.284708 18519 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:47.285048 18519 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:47.286978 18519 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/data/instance:
uuid: "b86a6b5ae577467eb35b046a0f5bb2ad"
format_stamp: "Formatted at 2026-08-12 08:03:47 on dist-test-slave-h3ft"
I20260812 08:03:47.287638 18519 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/wal/instance:
uuid: "b86a6b5ae577467eb35b046a0f5bb2ad"
format_stamp: "Formatted at 2026-08-12 08:03:47 on dist-test-slave-h3ft"
I20260812 08:03:47.292063 18519 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260812 08:03:47.295167 18546 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:47.296578 18519 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260812 08:03:47.296744 18519 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/wal
uuid: "b86a6b5ae577467eb35b046a0f5bb2ad"
format_stamp: "Formatted at 2026-08-12 08:03:47 on dist-test-slave-h3ft"
I20260812 08:03:47.296880 18519 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:47.330644 18519 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:47.331657 18519 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:47.331900 18519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:47.332685 18519 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:47.334015 18519 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:47.334095 18519 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:47.334177 18519 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:47.334230 18519 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:47.346247 18519 rpc_server.cc:307] RPC server started. Bound to: 127.15.57.129:32929
I20260812 08:03:47.346294 18720 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.57.129:32929 every 8 connection(s)
I20260812 08:03:47.347595 18519 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/data/info.pb
I20260812 08:03:47.353276 15590 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu as pid 18519
I20260812 08:03:47.353415 15590 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/wal/instance
I20260812 08:03:47.356213 15590 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
/tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.57.130:0
--local_ip_for_outbound_sockets=127.15.57.130
--webserver_interface=127.15.57.130
--webserver_port=0
--tserver_master_addrs=127.15.57.190:45861
--builtin_ntp_servers=127.15.57.148:40311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tablet_copy_fault_crash_on_fetch_all=1.0 with env {}
I20260812 08:03:47.359614 18721 heartbeater.cc:344] Connected to a master server at 127.15.57.190:45861
I20260812 08:03:47.360019 18721 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:47.360821 18721 heartbeater.cc:507] Master 127.15.57.190:45861 requested a full tablet report, sending...
I20260812 08:03:47.363339 18431 ts_manager.cc:194] Registered new tserver with Master: b86a6b5ae577467eb35b046a0f5bb2ad (127.15.57.129:32929)
I20260812 08:03:47.364939 18431 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.15.57.129:44857
W20260812 08:03:47.497107 18726 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:47.497452 18726 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:47.497530 18726 flags.cc:432] Enabled unsafe flag: --tablet_copy_fault_crash_on_fetch_all=1
W20260812 08:03:47.497591 18726 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:47.501453 18726 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:47.501662 18726 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.57.130
I20260812 08:03:47.506093 18726 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.57.148:40311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.57.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.57.130
--webserver_port=0
--tablet_copy_fault_crash_on_fetch_all=1
--tserver_master_addrs=127.15.57.190:45861
--never_fsync=true
--heap_profile_path=/tmp/kudu.18726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.57.130

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:47.507860 18726 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:47.509397 18726 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:47.518282 18736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:47.518304 18738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:47.518282 18740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:47.518720 18726 server_base.cc:1061] running on GCE node
I20260812 08:03:47.519284 18726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:47.519956 18726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:47.521159 18726 hybrid_clock.cc:648] HybridClock initialized: now 1786521827521139 us; error 33 us; skew 500 ppm
I20260812 08:03:47.523728 18726 webserver.cc:533] Webserver started at http://127.15.57.130:44135/ using document root <none> and password file <none>
I20260812 08:03:47.524605 18726 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:47.524703 18726 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:47.524979 18726 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:47.527158 18726 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/data/instance:
uuid: "2f3297543b0540c88240093f71e19142"
format_stamp: "Formatted at 2026-08-12 08:03:47 on dist-test-slave-h3ft"
I20260812 08:03:47.527781 18726 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/wal/instance:
uuid: "2f3297543b0540c88240093f71e19142"
format_stamp: "Formatted at 2026-08-12 08:03:47 on dist-test-slave-h3ft"
I20260812 08:03:47.532361 18726 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260812 08:03:47.535495 18746 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:47.536882 18726 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260812 08:03:47.537130 18726 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/wal
uuid: "2f3297543b0540c88240093f71e19142"
format_stamp: "Formatted at 2026-08-12 08:03:47 on dist-test-slave-h3ft"
I20260812 08:03:47.537281 18726 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:47.560564 18726 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:47.561570 18726 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:47.561849 18726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:47.562690 18726 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:47.564188 18726 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:47.564270 18726 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:47.564355 18726 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:47.564409 18726 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:47.576232 18726 rpc_server.cc:307] RPC server started. Bound to: 127.15.57.130:45465
I20260812 08:03:47.576341 18918 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.57.130:45465 every 8 connection(s)
I20260812 08:03:47.577636 18726 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/data/info.pb
I20260812 08:03:47.578671 15590 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu as pid 18726
I20260812 08:03:47.578804 15590 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/wal/instance
I20260812 08:03:47.582492 15590 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
/tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.57.131:0
--local_ip_for_outbound_sockets=127.15.57.131
--webserver_interface=127.15.57.131
--webserver_port=0
--tserver_master_addrs=127.15.57.190:45861
--builtin_ntp_servers=127.15.57.148:40311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tablet_copy_fault_crash_on_fetch_all=1.0 with env {}
I20260812 08:03:47.591243 18922 heartbeater.cc:344] Connected to a master server at 127.15.57.190:45861
I20260812 08:03:47.591673 18922 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:47.592689 18922 heartbeater.cc:507] Master 127.15.57.190:45861 requested a full tablet report, sending...
I20260812 08:03:47.594352 18431 ts_manager.cc:194] Registered new tserver with Master: 2f3297543b0540c88240093f71e19142 (127.15.57.130:45465)
I20260812 08:03:47.595208 18431 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.15.57.130:59165
W20260812 08:03:47.719794 18926 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:47.720146 18926 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:47.720230 18926 flags.cc:432] Enabled unsafe flag: --tablet_copy_fault_crash_on_fetch_all=1
W20260812 08:03:47.720290 18926 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:47.724030 18926 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:47.724263 18926 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.57.131
I20260812 08:03:47.728710 18926 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.57.148:40311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.57.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.57.131
--webserver_port=0
--tablet_copy_fault_crash_on_fetch_all=1
--tserver_master_addrs=127.15.57.190:45861
--never_fsync=true
--heap_profile_path=/tmp/kudu.18926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.57.131

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:47.730404 18926 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:47.732079 18926 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:47.741139 18935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:47.741175 18936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:47.741230 18926 server_base.cc:1061] running on GCE node
W20260812 08:03:47.741139 18946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:47.741845 18926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:47.742640 18926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:47.743892 18926 hybrid_clock.cc:648] HybridClock initialized: now 1786521827743850 us; error 54 us; skew 500 ppm
I20260812 08:03:47.746270 18926 webserver.cc:533] Webserver started at http://127.15.57.131:36651/ using document root <none> and password file <none>
I20260812 08:03:47.747442 18926 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:47.747517 18926 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:47.747743 18926 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:47.749843 18926 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-2/data/instance:
uuid: "d37ee240a1664042b3d420265ada87f8"
format_stamp: "Formatted at 2026-08-12 08:03:47 on dist-test-slave-h3ft"
I20260812 08:03:47.750420 18926 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-2/wal/instance:
uuid: "d37ee240a1664042b3d420265ada87f8"
format_stamp: "Formatted at 2026-08-12 08:03:47 on dist-test-slave-h3ft"
I20260812 08:03:47.754853 18926 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260812 08:03:47.757853 18953 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:47.759398 18926 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:03:47.759569 18926 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-2/wal
uuid: "d37ee240a1664042b3d420265ada87f8"
format_stamp: "Formatted at 2026-08-12 08:03:47 on dist-test-slave-h3ft"
I20260812 08:03:47.759713 18926 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:47.776242 18926 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:47.777275 18926 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:47.777570 18926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:47.778450 18926 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:47.780050 18926 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:47.780148 18926 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:47.780247 18926 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:47.780313 18926 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:47.794770 18926 rpc_server.cc:307] RPC server started. Bound to: 127.15.57.131:39311
I20260812 08:03:47.794800 19143 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.57.131:39311 every 8 connection(s)
I20260812 08:03:47.796361 18926 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-2/data/info.pb
I20260812 08:03:47.804126 15590 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu as pid 18926
I20260812 08:03:47.804308 15590 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-2/wal/instance
I20260812 08:03:47.807888 15590 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
/tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/wal
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/logs
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.57.132:0
--local_ip_for_outbound_sockets=127.15.57.132
--webserver_interface=127.15.57.132
--webserver_port=0
--tserver_master_addrs=127.15.57.190:45861
--builtin_ntp_servers=127.15.57.148:40311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tablet_copy_fault_crash_on_fetch_all=1.0 with env {}
I20260812 08:03:47.809468 19144 heartbeater.cc:344] Connected to a master server at 127.15.57.190:45861
I20260812 08:03:47.809854 19144 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:47.810809 19144 heartbeater.cc:507] Master 127.15.57.190:45861 requested a full tablet report, sending...
I20260812 08:03:47.812655 18431 ts_manager.cc:194] Registered new tserver with Master: d37ee240a1664042b3d420265ada87f8 (127.15.57.131:39311)
I20260812 08:03:47.813454 18431 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.15.57.131:55595
W20260812 08:03:47.969486 19147 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:47.969885 19147 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:47.969940 19147 flags.cc:432] Enabled unsafe flag: --tablet_copy_fault_crash_on_fetch_all=1
W20260812 08:03:47.969990 19147 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:47.974120 19147 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:47.974327 19147 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.57.132
I20260812 08:03:47.979333 19147 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.57.148:40311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/data
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.57.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/data/info.pb
--webserver_interface=127.15.57.132
--webserver_port=0
--tablet_copy_fault_crash_on_fetch_all=1
--tserver_master_addrs=127.15.57.190:45861
--never_fsync=true
--heap_profile_path=/tmp/kudu.19147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.57.132

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:47.981086 19147 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:47.982743 19147 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:47.991882 19161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:47.992100 19147 server_base.cc:1061] running on GCE node
W20260812 08:03:47.991971 19158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:47.991882 19159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:47.992698 19147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:47.993471 19147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:47.994688 19147 hybrid_clock.cc:648] HybridClock initialized: now 1786521827994668 us; error 42 us; skew 500 ppm
I20260812 08:03:47.997030 19147 webserver.cc:533] Webserver started at http://127.15.57.132:39259/ using document root <none> and password file <none>
I20260812 08:03:47.997802 19147 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:47.997911 19147 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:47.998204 19147 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:48.000372 19147 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/data/instance:
uuid: "884a923cef7e4941bee4372c54754fe6"
format_stamp: "Formatted at 2026-08-12 08:03:47 on dist-test-slave-h3ft"
I20260812 08:03:48.001046 19147 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/wal/instance:
uuid: "884a923cef7e4941bee4372c54754fe6"
format_stamp: "Formatted at 2026-08-12 08:03:47 on dist-test-slave-h3ft"
I20260812 08:03:48.005752 19147 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260812 08:03:48.008682 19168 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:48.010070 19147 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260812 08:03:48.010217 19147 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/data,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/wal
uuid: "884a923cef7e4941bee4372c54754fe6"
format_stamp: "Formatted at 2026-08-12 08:03:47 on dist-test-slave-h3ft"
I20260812 08:03:48.010349 19147 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/wal
metadata directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/wal
1 data directories: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:48.035349 19147 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:48.036293 19147 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:48.036537 19147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:48.037326 19147 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:48.038625 19147 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:48.038681 19147 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:48.038727 19147 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:48.038785 19147 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:48.050791 19147 rpc_server.cc:307] RPC server started. Bound to: 127.15.57.132:35759
I20260812 08:03:48.050817 19355 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.57.132:35759 every 8 connection(s)
I20260812 08:03:48.052135 19147 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/data/info.pb
I20260812 08:03:48.062134 15590 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu as pid 19147
I20260812 08:03:48.062282 15590 external_mini_cluster.cc:1448] Reading /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/wal/instance
I20260812 08:03:48.064321 19357 heartbeater.cc:344] Connected to a master server at 127.15.57.190:45861
I20260812 08:03:48.064695 19357 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:48.065546 19357 heartbeater.cc:507] Master 127.15.57.190:45861 requested a full tablet report, sending...
I20260812 08:03:48.067078 18429 ts_manager.cc:194] Registered new tserver with Master: 884a923cef7e4941bee4372c54754fe6 (127.15.57.132:35759)
I20260812 08:03:48.067864 18429 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.15.57.132:46495
I20260812 08:03:48.078050 15590 external_mini_cluster.cc:955] 4 TS(s) registered with all masters
I20260812 08:03:48.100780 18429 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53232:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260812 08:03:48.124061 19038 tablet_service.cc:1511] Processing CreateTablet for tablet bfc834b79d014cf19efba2554c569556 (DEFAULT_TABLE table=test-workload [id=124fc5eeb024453987101834babd06a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:48.124053 18627 tablet_service.cc:1511] Processing CreateTablet for tablet bfc834b79d014cf19efba2554c569556 (DEFAULT_TABLE table=test-workload [id=124fc5eeb024453987101834babd06a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:48.124046 19249 tablet_service.cc:1511] Processing CreateTablet for tablet bfc834b79d014cf19efba2554c569556 (DEFAULT_TABLE table=test-workload [id=124fc5eeb024453987101834babd06a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 08:03:48.125379 19249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfc834b79d014cf19efba2554c569556. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:48.125379 19038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfc834b79d014cf19efba2554c569556. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:48.125379 18627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfc834b79d014cf19efba2554c569556. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:48.131708 19394 tablet_bootstrap.cc:492] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6: Bootstrap starting.
I20260812 08:03:48.132931 19398 tablet_bootstrap.cc:492] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad: Bootstrap starting.
I20260812 08:03:48.133352 19397 tablet_bootstrap.cc:492] T bfc834b79d014cf19efba2554c569556 P d37ee240a1664042b3d420265ada87f8: Bootstrap starting.
I20260812 08:03:48.135252 19394 tablet_bootstrap.cc:654] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:48.135255 19398 tablet_bootstrap.cc:654] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:48.135569 19397 tablet_bootstrap.cc:654] T bfc834b79d014cf19efba2554c569556 P d37ee240a1664042b3d420265ada87f8: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:48.136329 19397 log.cc:826] T bfc834b79d014cf19efba2554c569556 P d37ee240a1664042b3d420265ada87f8: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:48.136319 19394 log.cc:826] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:48.136330 19398 log.cc:826] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:48.138374 19394 tablet_bootstrap.cc:492] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6: No bootstrap required, opened a new log
I20260812 08:03:48.138365 19397 tablet_bootstrap.cc:492] T bfc834b79d014cf19efba2554c569556 P d37ee240a1664042b3d420265ada87f8: No bootstrap required, opened a new log
I20260812 08:03:48.138373 19398 tablet_bootstrap.cc:492] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad: No bootstrap required, opened a new log
I20260812 08:03:48.138598 19397 ts_tablet_manager.cc:1403] T bfc834b79d014cf19efba2554c569556 P d37ee240a1664042b3d420265ada87f8: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 08:03:48.138598 19394 ts_tablet_manager.cc:1403] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6: Time spent bootstrapping tablet: real 0.007s	user 0.005s	sys 0.000s
I20260812 08:03:48.138616 19398 ts_tablet_manager.cc:1403] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad: Time spent bootstrapping tablet: real 0.006s	user 0.000s	sys 0.004s
I20260812 08:03:48.141935 19398 raft_consensus.cc:359] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 32929 } } peers { permanent_uuid: "d37ee240a1664042b3d420265ada87f8" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 39311 } } peers { permanent_uuid: "884a923cef7e4941bee4372c54754fe6" member_type: VOTER last_known_addr { host: "127.15.57.132" port: 35759 } }
I20260812 08:03:48.142048 19397 raft_consensus.cc:359] T bfc834b79d014cf19efba2554c569556 P d37ee240a1664042b3d420265ada87f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 32929 } } peers { permanent_uuid: "d37ee240a1664042b3d420265ada87f8" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 39311 } } peers { permanent_uuid: "884a923cef7e4941bee4372c54754fe6" member_type: VOTER last_known_addr { host: "127.15.57.132" port: 35759 } }
I20260812 08:03:48.142050 19394 raft_consensus.cc:359] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 32929 } } peers { permanent_uuid: "d37ee240a1664042b3d420265ada87f8" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 39311 } } peers { permanent_uuid: "884a923cef7e4941bee4372c54754fe6" member_type: VOTER last_known_addr { host: "127.15.57.132" port: 35759 } }
I20260812 08:03:48.142231 19398 raft_consensus.cc:385] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:48.142261 19397 raft_consensus.cc:385] T bfc834b79d014cf19efba2554c569556 P d37ee240a1664042b3d420265ada87f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:48.142261 19394 raft_consensus.cc:385] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:48.142328 19397 raft_consensus.cc:740] T bfc834b79d014cf19efba2554c569556 P d37ee240a1664042b3d420265ada87f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d37ee240a1664042b3d420265ada87f8, State: Initialized, Role: FOLLOWER
I20260812 08:03:48.142326 19398 raft_consensus.cc:740] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b86a6b5ae577467eb35b046a0f5bb2ad, State: Initialized, Role: FOLLOWER
I20260812 08:03:48.142493 19394 raft_consensus.cc:740] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884a923cef7e4941bee4372c54754fe6, State: Initialized, Role: FOLLOWER
I20260812 08:03:48.143023 19394 consensus_queue.cc:260] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 32929 } } peers { permanent_uuid: "d37ee240a1664042b3d420265ada87f8" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 39311 } } peers { permanent_uuid: "884a923cef7e4941bee4372c54754fe6" member_type: VOTER last_known_addr { host: "127.15.57.132" port: 35759 } }
I20260812 08:03:48.143021 19397 consensus_queue.cc:260] T bfc834b79d014cf19efba2554c569556 P d37ee240a1664042b3d420265ada87f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 32929 } } peers { permanent_uuid: "d37ee240a1664042b3d420265ada87f8" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 39311 } } peers { permanent_uuid: "884a923cef7e4941bee4372c54754fe6" member_type: VOTER last_known_addr { host: "127.15.57.132" port: 35759 } }
I20260812 08:03:48.143003 19398 consensus_queue.cc:260] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 32929 } } peers { permanent_uuid: "d37ee240a1664042b3d420265ada87f8" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 39311 } } peers { permanent_uuid: "884a923cef7e4941bee4372c54754fe6" member_type: VOTER last_known_addr { host: "127.15.57.132" port: 35759 } }
I20260812 08:03:48.144171 19397 ts_tablet_manager.cc:1434] T bfc834b79d014cf19efba2554c569556 P d37ee240a1664042b3d420265ada87f8: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260812 08:03:48.144171 19394 ts_tablet_manager.cc:1434] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260812 08:03:48.144174 19398 ts_tablet_manager.cc:1434] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.003s
I20260812 08:03:48.144634 19144 heartbeater.cc:499] Master 127.15.57.190:45861 was elected leader, sending a full tablet report...
I20260812 08:03:48.144637 18721 heartbeater.cc:499] Master 127.15.57.190:45861 was elected leader, sending a full tablet report...
I20260812 08:03:48.145993 19357 heartbeater.cc:499] Master 127.15.57.190:45861 was elected leader, sending a full tablet report...
I20260812 08:03:48.162254 19404 raft_consensus.cc:493] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:48.162560 19404 raft_consensus.cc:515] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 32929 } } peers { permanent_uuid: "d37ee240a1664042b3d420265ada87f8" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 39311 } } peers { permanent_uuid: "884a923cef7e4941bee4372c54754fe6" member_type: VOTER last_known_addr { host: "127.15.57.132" port: 35759 } }
I20260812 08:03:48.164175 19404 leader_election.cc:290] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d37ee240a1664042b3d420265ada87f8 (127.15.57.131:39311), 884a923cef7e4941bee4372c54754fe6 (127.15.57.132:35759)
I20260812 08:03:48.170459 19283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc834b79d014cf19efba2554c569556" candidate_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884a923cef7e4941bee4372c54754fe6" is_pre_election: true
I20260812 08:03:48.170835 19070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc834b79d014cf19efba2554c569556" candidate_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d37ee240a1664042b3d420265ada87f8" is_pre_election: true
I20260812 08:03:48.170960 19283 raft_consensus.cc:2468] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b86a6b5ae577467eb35b046a0f5bb2ad in term 0.
I20260812 08:03:48.171165 19070 raft_consensus.cc:2468] T bfc834b79d014cf19efba2554c569556 P d37ee240a1664042b3d420265ada87f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b86a6b5ae577467eb35b046a0f5bb2ad in term 0.
I20260812 08:03:48.171710 18548 leader_election.cc:304] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 884a923cef7e4941bee4372c54754fe6, b86a6b5ae577467eb35b046a0f5bb2ad, d37ee240a1664042b3d420265ada87f8; no voters: 
I20260812 08:03:48.172153 19404 raft_consensus.cc:2804] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:03:48.172235 19404 raft_consensus.cc:493] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:03:48.172266 19404 raft_consensus.cc:3060] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:48.173419 19404 raft_consensus.cc:515] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 32929 } } peers { permanent_uuid: "d37ee240a1664042b3d420265ada87f8" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 39311 } } peers { permanent_uuid: "884a923cef7e4941bee4372c54754fe6" member_type: VOTER last_known_addr { host: "127.15.57.132" port: 35759 } }
I20260812 08:03:48.173900 19404 leader_election.cc:290] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [CANDIDATE]: Term 1 election: Requested vote from peers d37ee240a1664042b3d420265ada87f8 (127.15.57.131:39311), 884a923cef7e4941bee4372c54754fe6 (127.15.57.132:35759)
I20260812 08:03:48.174412 19283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc834b79d014cf19efba2554c569556" candidate_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "884a923cef7e4941bee4372c54754fe6"
I20260812 08:03:48.174410 19070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc834b79d014cf19efba2554c569556" candidate_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d37ee240a1664042b3d420265ada87f8"
I20260812 08:03:48.174566 19283 raft_consensus.cc:3060] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:48.174566 19070 raft_consensus.cc:3060] T bfc834b79d014cf19efba2554c569556 P d37ee240a1664042b3d420265ada87f8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:48.176160 19070 raft_consensus.cc:2468] T bfc834b79d014cf19efba2554c569556 P d37ee240a1664042b3d420265ada87f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b86a6b5ae577467eb35b046a0f5bb2ad in term 1.
I20260812 08:03:48.176551 19283 raft_consensus.cc:2468] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b86a6b5ae577467eb35b046a0f5bb2ad in term 1.
I20260812 08:03:48.176618 18552 leader_election.cc:304] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b86a6b5ae577467eb35b046a0f5bb2ad, d37ee240a1664042b3d420265ada87f8; no voters: 
I20260812 08:03:48.176821 19404 raft_consensus.cc:2804] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:48.177168 19404 raft_consensus.cc:697] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [term 1 LEADER]: Becoming Leader. State: Replica: b86a6b5ae577467eb35b046a0f5bb2ad, State: Running, Role: LEADER
I20260812 08:03:48.177618 19404 consensus_queue.cc:237] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 32929 } } peers { permanent_uuid: "d37ee240a1664042b3d420265ada87f8" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 39311 } } peers { permanent_uuid: "884a923cef7e4941bee4372c54754fe6" member_type: VOTER last_known_addr { host: "127.15.57.132" port: 35759 } }
I20260812 08:03:48.181839 18427 catalog_manager.cc:5719] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad reported cstate change: term changed from 0 to 1, leader changed from <none> to b86a6b5ae577467eb35b046a0f5bb2ad (127.15.57.129). New cstate: current_term: 1 leader_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 32929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d37ee240a1664042b3d420265ada87f8" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 39311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "884a923cef7e4941bee4372c54754fe6" member_type: VOTER last_known_addr { host: "127.15.57.132" port: 35759 } health_report { overall_health: UNKNOWN } } }
W20260812 08:03:48.297998 19145 tablet.cc:2406] T bfc834b79d014cf19efba2554c569556 P d37ee240a1664042b3d420265ada87f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:03:48.303285 19359 tablet.cc:2406] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 08:03:48.349671 18724 tablet.cc:2406] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:03:48.598608 18922 heartbeater.cc:499] Master 127.15.57.190:45861 was elected leader, sending a full tablet report...
I20260812 08:03:48.657694 19404 consensus_queue.cc:1048] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "d37ee240a1664042b3d420265ada87f8" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 39311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:03:48.664785 19420 consensus_queue.cc:1048] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "884a923cef7e4941bee4372c54754fe6" member_type: VOTER last_known_addr { host: "127.15.57.132" port: 35759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:03:48.671370 18657 consensus_queue.cc:237] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 32929 } } peers { permanent_uuid: "d37ee240a1664042b3d420265ada87f8" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 39311 } } peers { permanent_uuid: "884a923cef7e4941bee4372c54754fe6" member_type: VOTER last_known_addr { host: "127.15.57.132" port: 35759 } } peers { permanent_uuid: "2f3297543b0540c88240093f71e19142" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 45465 } attrs { } }
I20260812 08:03:48.673249 19283 raft_consensus.cc:1275] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 [term 1 FOLLOWER]: Refusing update from remote peer b86a6b5ae577467eb35b046a0f5bb2ad: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:03:48.673249 19070 raft_consensus.cc:1275] T bfc834b79d014cf19efba2554c569556 P d37ee240a1664042b3d420265ada87f8 [term 1 FOLLOWER]: Refusing update from remote peer b86a6b5ae577467eb35b046a0f5bb2ad: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:03:48.673816 19409 consensus_queue.cc:1048] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "884a923cef7e4941bee4372c54754fe6" member_type: VOTER last_known_addr { host: "127.15.57.132" port: 35759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:48.674003 19420 consensus_queue.cc:1048] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "d37ee240a1664042b3d420265ada87f8" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 39311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 08:03:48.676072 19420 raft_consensus.cc:2955] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 2f3297543b0540c88240093f71e19142 (127.15.57.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 32929 } } peers { permanent_uuid: "d37ee240a1664042b3d420265ada87f8" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 39311 } } peers { permanent_uuid: "884a923cef7e4941bee4372c54754fe6" member_type: VOTER last_known_addr { host: "127.15.57.132" port: 35759 } } peers { permanent_uuid: "2f3297543b0540c88240093f71e19142" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 45465 } attrs { } } }
I20260812 08:03:48.676646 19070 raft_consensus.cc:2955] T bfc834b79d014cf19efba2554c569556 P d37ee240a1664042b3d420265ada87f8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 2f3297543b0540c88240093f71e19142 (127.15.57.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 32929 } } peers { permanent_uuid: "d37ee240a1664042b3d420265ada87f8" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 39311 } } peers { permanent_uuid: "884a923cef7e4941bee4372c54754fe6" member_type: VOTER last_known_addr { host: "127.15.57.132" port: 35759 } } peers { permanent_uuid: "2f3297543b0540c88240093f71e19142" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 45465 } attrs { } } }
W20260812 08:03:48.678974 18547 consensus_peers.cc:597] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad -> Peer 2f3297543b0540c88240093f71e19142 (127.15.57.130:45465): Couldn't send request to peer 2f3297543b0540c88240093f71e19142. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bfc834b79d014cf19efba2554c569556. This is attempt 1: this message will repeat every 5th retry.
I20260812 08:03:48.680065 19283 raft_consensus.cc:2955] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 2f3297543b0540c88240093f71e19142 (127.15.57.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 32929 } } peers { permanent_uuid: "d37ee240a1664042b3d420265ada87f8" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 39311 } } peers { permanent_uuid: "884a923cef7e4941bee4372c54754fe6" member_type: VOTER last_known_addr { host: "127.15.57.132" port: 35759 } } peers { permanent_uuid: "2f3297543b0540c88240093f71e19142" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 45465 } attrs { } } }
I20260812 08:03:48.680431 18431 catalog_manager.cc:5719] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad reported cstate change: config changed from index -1 to 2, VOTER 2f3297543b0540c88240093f71e19142 (127.15.57.130) added. New cstate: current_term: 1 leader_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 32929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d37ee240a1664042b3d420265ada87f8" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 39311 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "884a923cef7e4941bee4372c54754fe6" member_type: VOTER last_known_addr { host: "127.15.57.132" port: 35759 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f3297543b0540c88240093f71e19142" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 45465 } attrs { } health_report { overall_health: UNKNOWN } } }
I20260812 08:03:49.290856 19430 ts_tablet_manager.cc:933] T bfc834b79d014cf19efba2554c569556 P 2f3297543b0540c88240093f71e19142: Initiating tablet copy from peer b86a6b5ae577467eb35b046a0f5bb2ad (127.15.57.129:32929)
I20260812 08:03:49.291657 19430 tablet_copy_client.cc:323] T bfc834b79d014cf19efba2554c569556 P 2f3297543b0540c88240093f71e19142: tablet copy: Beginning tablet copy session from remote peer at address 127.15.57.129:32929
I20260812 08:03:49.296329 18684 tablet_copy_service.cc:140] P b86a6b5ae577467eb35b046a0f5bb2ad: Received BeginTabletCopySession request for tablet bfc834b79d014cf19efba2554c569556 from peer 2f3297543b0540c88240093f71e19142 ({username='slave'} at 127.15.57.130:56157)
I20260812 08:03:49.296541 18684 tablet_copy_service.cc:161] P b86a6b5ae577467eb35b046a0f5bb2ad: Beginning new tablet copy session on tablet bfc834b79d014cf19efba2554c569556 from peer 2f3297543b0540c88240093f71e19142 at {username='slave'} at 127.15.57.130:56157: session id = 2f3297543b0540c88240093f71e19142-bfc834b79d014cf19efba2554c569556
I20260812 08:03:49.298036 18684 tablet_copy_source_session.cc:215] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 08:03:49.302896 19430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfc834b79d014cf19efba2554c569556. 1 dirs total, 0 dirs full, 0 dirs failed
W20260812 08:03:49.308909 19430 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20260812 08:03:49.309005 19430 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20260812 08:03:49.309034 19430 fault_injection.cc:59] injecting fault for kudu (pid 18726): FLAGS_tablet_copy_fault_crash_on_fetch_all (process will exit)
I20260812 08:03:49.711234 15590 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu with pid 18519
I20260812 08:03:49.733059 15590 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu with pid 18926
I20260812 08:03:49.749569 15590 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu with pid 19147
I20260812 08:03:49.772928 15590 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu with pid 18377
I20260812 08:03:49.794164 15590 tablet_copy-itest.cc:1348] Restarting tablet servers. New replica TS UUID: 2f3297543b0540c88240093f71e19142, tablet data state: TABLET_DATA_COPYING, last-logged opid: 1.0
I20260812 08:03:49.794704 15590 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
/tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.57.190:45861
--webserver_interface=127.15.57.190
--webserver_port=43187
--builtin_ntp_servers=127.15.57.148:40311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.57.190:45861
--catalog_manager_evict_excess_replicas=false
--master_add_server_when_underreplicated=false with env {}
W20260812 08:03:49.931252 19437 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:49.931624 19437 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:49.931705 19437 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:49.935642 19437 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:03:49.935757 19437 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:49.935796 19437 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:03:49.935837 19437 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 08:03:49.940253 19437 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.57.148:40311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/wal
--catalog_manager_evict_excess_replicas=false
--master_add_server_when_underreplicated=false
--ipki_ca_key_size=768
--master_addresses=127.15.57.190:45861
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.57.190:45861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.57.190
--webserver_port=43187
--never_fsync=true
--heap_profile_path=/tmp/kudu.19437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:49.941903 19437 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:49.943643 19437 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:49.950909 19447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:49.950886 19448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:49.950974 19456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:49.951355 19437 server_base.cc:1061] running on GCE node
I20260812 08:03:49.951992 19437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:49.953342 19437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:49.954521 19437 hybrid_clock.cc:648] HybridClock initialized: now 1786521829954482 us; error 55 us; skew 500 ppm
I20260812 08:03:49.957082 19437 webserver.cc:533] Webserver started at http://127.15.57.190:43187/ using document root <none> and password file <none>
I20260812 08:03:49.958382 19437 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:49.958478 19437 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:49.962823 19437 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.002s	sys 0.001s
I20260812 08:03:49.965472 19463 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:49.966801 19437 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260812 08:03:49.966982 19437 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/data,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/wal
uuid: "f64aa79d48dc45b7ab0ee33054daa8de"
format_stamp: "Formatted at 2026-08-12 08:03:46 on dist-test-slave-h3ft"
I20260812 08:03:49.967545 19437 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:50.004469 19437 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:50.005584 19437 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:50.005862 19437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:50.016295 19437 rpc_server.cc:307] RPC server started. Bound to: 127.15.57.190:45861
I20260812 08:03:50.016312 19548 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.57.190:45861 every 8 connection(s)
I20260812 08:03:50.017825 19437 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/master-0/data/info.pb
I20260812 08:03:50.023337 19549 sys_catalog.cc:263] Verifying existing consensus state
I20260812 08:03:50.024576 15590 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu as pid 19437
I20260812 08:03:50.025198 15590 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
/tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.57.130:45465
--local_ip_for_outbound_sockets=127.15.57.130
--tserver_master_addrs=127.15.57.190:45861
--webserver_port=44135
--webserver_interface=127.15.57.130
--builtin_ntp_servers=127.15.57.148:40311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260812 08:03:50.025518 19549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de: Bootstrap starting.
I20260812 08:03:50.044564 19549 log.cc:826] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:50.054621 19549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=3 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:03:50.055215 19549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de: Bootstrap complete.
I20260812 08:03:50.059563 19549 raft_consensus.cc:359] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f64aa79d48dc45b7ab0ee33054daa8de" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 45861 } }
I20260812 08:03:50.059989 19549 raft_consensus.cc:740] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f64aa79d48dc45b7ab0ee33054daa8de, State: Initialized, Role: FOLLOWER
I20260812 08:03:50.060693 19549 consensus_queue.cc:260] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f64aa79d48dc45b7ab0ee33054daa8de" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 45861 } }
I20260812 08:03:50.060889 19549 raft_consensus.cc:399] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:03:50.060993 19549 raft_consensus.cc:493] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:03:50.061141 19549 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:03:50.063117 19549 raft_consensus.cc:515] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f64aa79d48dc45b7ab0ee33054daa8de" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 45861 } }
I20260812 08:03:50.063568 19549 leader_election.cc:304] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f64aa79d48dc45b7ab0ee33054daa8de; no voters: 
I20260812 08:03:50.064023 19549 leader_election.cc:290] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [CANDIDATE]: Term 2 election: Requested vote from peers 
I20260812 08:03:50.064153 19553 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [term 2 FOLLOWER]: Leader election won for term 2
I20260812 08:03:50.064373 19553 raft_consensus.cc:697] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [term 2 LEADER]: Becoming Leader. State: Replica: f64aa79d48dc45b7ab0ee33054daa8de, State: Running, Role: LEADER
I20260812 08:03:50.064728 19553 consensus_queue.cc:237] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f64aa79d48dc45b7ab0ee33054daa8de" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 45861 } }
I20260812 08:03:50.065532 19549 sys_catalog.cc:565] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:03:50.066421 19554 sys_catalog.cc:455] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f64aa79d48dc45b7ab0ee33054daa8de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f64aa79d48dc45b7ab0ee33054daa8de" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 45861 } } }
I20260812 08:03:50.066452 19555 sys_catalog.cc:455] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [sys.catalog]: SysCatalogTable state changed. Reason: New leader f64aa79d48dc45b7ab0ee33054daa8de. Latest consensus state: current_term: 2 leader_uuid: "f64aa79d48dc45b7ab0ee33054daa8de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f64aa79d48dc45b7ab0ee33054daa8de" member_type: VOTER last_known_addr { host: "127.15.57.190" port: 45861 } } }
I20260812 08:03:50.066548 19554 sys_catalog.cc:458] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:50.066570 19555 sys_catalog.cc:458] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:50.067055 19558 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:03:50.071777 19558 catalog_manager.cc:679] Loaded metadata for table test-workload [id=124fc5eeb024453987101834babd06a7]
I20260812 08:03:50.073761 19558 tablet_loader.cc:96] loaded metadata for tablet bfc834b79d014cf19efba2554c569556 (table test-workload [id=124fc5eeb024453987101834babd06a7])
I20260812 08:03:50.073966 19558 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:03:50.074862 19558 catalog_manager.cc:1295] Loaded cluster ID: 30b1224f99d7457a925b4463860fa739
I20260812 08:03:50.074935 19558 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:03:50.077625 19558 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:03:50.078469 19558 catalog_manager.cc:6103] T 00000000000000000000000000000000 P f64aa79d48dc45b7ab0ee33054daa8de: Loaded TSK: 0
I20260812 08:03:50.079718 19558 catalog_manager.cc:1550] Initializing in-progress tserver states...
W20260812 08:03:50.164342 19551 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:50.164647 19551 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:50.164729 19551 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:50.168521 19551 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:50.168691 19551 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.57.130
I20260812 08:03:50.172904 19551 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.57.148:40311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.57.130:45465
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.57.130
--webserver_port=44135
--tserver_master_addrs=127.15.57.190:45861
--never_fsync=true
--heap_profile_path=/tmp/kudu.19551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.57.130

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:50.174422 19551 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:50.175930 19551 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:50.183873 19596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:50.184017 19551 server_base.cc:1061] running on GCE node
W20260812 08:03:50.183785 19594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:50.183800 19592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:50.184598 19551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:50.185322 19551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:50.186511 19551 hybrid_clock.cc:648] HybridClock initialized: now 1786521830186497 us; error 46 us; skew 500 ppm
I20260812 08:03:50.188845 19551 webserver.cc:533] Webserver started at http://127.15.57.130:44135/ using document root <none> and password file <none>
I20260812 08:03:50.189566 19551 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:50.189677 19551 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:50.193878 19551 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.001s	sys 0.001s
I20260812 08:03:50.196354 19610 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:50.197624 19551 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:03:50.197772 19551 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/wal
uuid: "2f3297543b0540c88240093f71e19142"
format_stamp: "Formatted at 2026-08-12 08:03:47 on dist-test-slave-h3ft"
I20260812 08:03:50.198256 19551 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:50.212227 19551 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:50.213143 19551 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:50.213388 19551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:50.214236 19551 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:50.215950 19619 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260812 08:03:50.220393 19619 ts_tablet_manager.cc:1863] T bfc834b79d014cf19efba2554c569556 P 2f3297543b0540c88240093f71e19142: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_COPYING
I20260812 08:03:50.221750 19619 ts_tablet_manager.cc:1916] T bfc834b79d014cf19efba2554c569556 P 2f3297543b0540c88240093f71e19142: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 08:03:50.225878 19619 ts_tablet_manager.cc:1929] T bfc834b79d014cf19efba2554c569556 P 2f3297543b0540c88240093f71e19142: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.0
I20260812 08:03:50.226960 19551 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260812 08:03:50.227083 19551 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.011s	user 0.001s	sys 0.000s
I20260812 08:03:50.227190 19551 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:50.227222 19551 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:50.241098 19551 rpc_server.cc:307] RPC server started. Bound to: 127.15.57.130:45465
I20260812 08:03:50.241122 19801 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.57.130:45465 every 8 connection(s)
I20260812 08:03:50.242453 19551 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-1/data/info.pb
I20260812 08:03:50.245128 15590 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu as pid 19551
I20260812 08:03:50.245743 15590 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
/tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/wal
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/logs
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.57.132:35759
--local_ip_for_outbound_sockets=127.15.57.132
--tserver_master_addrs=127.15.57.190:45861
--webserver_port=39259
--webserver_interface=127.15.57.132
--builtin_ntp_servers=127.15.57.148:40311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260812 08:03:50.257884 19802 heartbeater.cc:344] Connected to a master server at 127.15.57.190:45861
I20260812 08:03:50.258275 19802 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:50.259238 19802 heartbeater.cc:507] Master 127.15.57.190:45861 requested a full tablet report, sending...
I20260812 08:03:50.261871 19489 ts_manager.cc:194] Registered new tserver with Master: 2f3297543b0540c88240093f71e19142 (127.15.57.130:45465)
I20260812 08:03:50.264425 19489 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.15.57.130:43473
W20260812 08:03:50.378424 19807 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:50.378728 19807 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:50.378778 19807 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:50.382454 19807 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:50.382578 19807 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.57.132
I20260812 08:03:50.386788 19807 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.57.148:40311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/data
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.57.132:35759
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/data/info.pb
--webserver_interface=127.15.57.132
--webserver_port=39259
--tserver_master_addrs=127.15.57.190:45861
--never_fsync=true
--heap_profile_path=/tmp/kudu.19807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.57.132

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:50.388336 19807 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:50.389756 19807 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:50.397943 19821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:50.397933 19818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:50.397933 19819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:50.398432 19807 server_base.cc:1061] running on GCE node
I20260812 08:03:50.398866 19807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:50.399574 19807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:50.400779 19807 hybrid_clock.cc:648] HybridClock initialized: now 1786521830400762 us; error 35 us; skew 500 ppm
I20260812 08:03:50.402966 19807 webserver.cc:533] Webserver started at http://127.15.57.132:39259/ using document root <none> and password file <none>
I20260812 08:03:50.403795 19807 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:50.403882 19807 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:50.407941 19807 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.001s	sys 0.003s
I20260812 08:03:50.410243 19834 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:50.411729 19807 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:03:50.411881 19807 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/data,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/wal
uuid: "884a923cef7e4941bee4372c54754fe6"
format_stamp: "Formatted at 2026-08-12 08:03:47 on dist-test-slave-h3ft"
I20260812 08:03:50.412366 19807 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/wal
metadata directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/wal
1 data directories: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:50.427572 19807 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:50.428414 19807 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:50.428655 19807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:50.429363 19807 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:50.430902 19844 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:03:50.435248 19807 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:03:50.435333 19807 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.005s	user 0.000s	sys 0.000s
I20260812 08:03:50.435459 19807 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:03:50.437311 19807 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:03:50.437390 19807 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 08:03:50.437829 19844 tablet_bootstrap.cc:492] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6: Bootstrap starting.
I20260812 08:03:50.451628 19807 rpc_server.cc:307] RPC server started. Bound to: 127.15.57.132:35759
I20260812 08:03:50.451663 20031 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.57.132:35759 every 8 connection(s)
I20260812 08:03:50.453055 19807 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-3/data/info.pb
I20260812 08:03:50.456681 19844 log.cc:826] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:50.456866 15590 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu as pid 19807
I20260812 08:03:50.457533 15590 external_mini_cluster.cc:1372] Running /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
/tmp/dist-test-tasklDZR8P/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.57.129:32929
--local_ip_for_outbound_sockets=127.15.57.129
--tserver_master_addrs=127.15.57.190:45861
--webserver_port=38195
--webserver_interface=127.15.57.129
--builtin_ntp_servers=127.15.57.148:40311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20260812 08:03:50.461392 19844 tablet_bootstrap.cc:492] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:03:50.461874 19844 tablet_bootstrap.cc:492] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6: Bootstrap complete.
I20260812 08:03:50.463629 19844 ts_tablet_manager.cc:1403] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6: Time spent bootstrapping tablet: real 0.026s	user 0.019s	sys 0.007s
I20260812 08:03:50.464404 20033 heartbeater.cc:344] Connected to a master server at 127.15.57.190:45861
I20260812 08:03:50.464722 20033 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:50.465499 20033 heartbeater.cc:507] Master 127.15.57.190:45861 requested a full tablet report, sending...
I20260812 08:03:50.468075 19489 ts_manager.cc:194] Registered new tserver with Master: 884a923cef7e4941bee4372c54754fe6 (127.15.57.132:35759)
I20260812 08:03:50.468386 19844 raft_consensus.cc:359] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 32929 } } peers { permanent_uuid: "d37ee240a1664042b3d420265ada87f8" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 39311 } } peers { permanent_uuid: "884a923cef7e4941bee4372c54754fe6" member_type: VOTER last_known_addr { host: "127.15.57.132" port: 35759 } } peers { permanent_uuid: "2f3297543b0540c88240093f71e19142" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 45465 } attrs { } }
I20260812 08:03:50.469002 19844 raft_consensus.cc:740] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 884a923cef7e4941bee4372c54754fe6, State: Initialized, Role: FOLLOWER
I20260812 08:03:50.469344 19489 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.15.57.132:56259
I20260812 08:03:50.469692 19844 consensus_queue.cc:260] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 32929 } } peers { permanent_uuid: "d37ee240a1664042b3d420265ada87f8" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 39311 } } peers { permanent_uuid: "884a923cef7e4941bee4372c54754fe6" member_type: VOTER last_known_addr { host: "127.15.57.132" port: 35759 } } peers { permanent_uuid: "2f3297543b0540c88240093f71e19142" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 45465 } attrs { } }
I20260812 08:03:50.471110 19844 ts_tablet_manager.cc:1434] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260812 08:03:50.473232 20033 heartbeater.cc:499] Master 127.15.57.190:45861 was elected leader, sending a full tablet report...
W20260812 08:03:50.595562 20042 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:50.595887 20042 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:50.595978 20042 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:50.599689 20042 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:50.599882 20042 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.57.129
I20260812 08:03:50.604359 20042 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.57.148:40311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.57.129:32929
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.57.129
--webserver_port=38195
--tserver_master_addrs=127.15.57.190:45861
--never_fsync=true
--heap_profile_path=/tmp/kudu.20042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.57.129

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:50.606020 20042 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:50.607679 20042 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:50.616531 20055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:50.616676 20042 server_base.cc:1061] running on GCE node
W20260812 08:03:50.616533 20050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:50.616531 20052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:50.617343 20042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:50.618052 20042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:50.619240 20042 hybrid_clock.cc:648] HybridClock initialized: now 1786521830619226 us; error 41 us; skew 500 ppm
I20260812 08:03:50.621349 20042 webserver.cc:533] Webserver started at http://127.15.57.129:38195/ using document root <none> and password file <none>
I20260812 08:03:50.622089 20042 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:50.622207 20042 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:50.626413 20042 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.002s	sys 0.000s
I20260812 08:03:50.629030 20067 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:50.630297 20042 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260812 08:03:50.630460 20042 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/wal
uuid: "b86a6b5ae577467eb35b046a0f5bb2ad"
format_stamp: "Formatted at 2026-08-12 08:03:47 on dist-test-slave-h3ft"
I20260812 08:03:50.630952 20042 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:50.667281 20042 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:50.668336 20042 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:50.668581 20042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:50.669369 20042 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:50.670984 20077 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 08:03:50.675509 20042 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 08:03:50.675603 20042 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.005s	user 0.001s	sys 0.000s
I20260812 08:03:50.675735 20042 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 08:03:50.677614 20042 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 08:03:50.677686 20042 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 08:03:50.678112 20077 tablet_bootstrap.cc:492] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad: Bootstrap starting.
I20260812 08:03:50.691710 20042 rpc_server.cc:307] RPC server started. Bound to: 127.15.57.129:32929
I20260812 08:03:50.691761 20268 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.57.129:32929 every 8 connection(s)
I20260812 08:03:50.693169 20042 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasklDZR8P/test-tmp/tablet_copy-itest.2.FailureCause_TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote_1.1786521813119228-15590-0/minicluster-data/ts-0/data/info.pb
I20260812 08:03:50.697882 20077 log.cc:826] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:50.700119 15590 external_mini_cluster.cc:1434] Started /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu as pid 20042
I20260812 08:03:50.701788 20077 tablet_bootstrap.cc:492] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:03:50.702375 20077 tablet_bootstrap.cc:492] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad: Bootstrap complete.
I20260812 08:03:50.704229 20077 ts_tablet_manager.cc:1403] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.011s
I20260812 08:03:50.706311 20269 heartbeater.cc:344] Connected to a master server at 127.15.57.190:45861
I20260812 08:03:50.706631 20269 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:50.708148 20077 raft_consensus.cc:359] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 32929 } } peers { permanent_uuid: "d37ee240a1664042b3d420265ada87f8" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 39311 } } peers { permanent_uuid: "884a923cef7e4941bee4372c54754fe6" member_type: VOTER last_known_addr { host: "127.15.57.132" port: 35759 } } peers { permanent_uuid: "2f3297543b0540c88240093f71e19142" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 45465 } attrs { } }
I20260812 08:03:50.708701 20077 raft_consensus.cc:740] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b86a6b5ae577467eb35b046a0f5bb2ad, State: Initialized, Role: FOLLOWER
I20260812 08:03:50.709442 20077 consensus_queue.cc:260] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 32929 } } peers { permanent_uuid: "d37ee240a1664042b3d420265ada87f8" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 39311 } } peers { permanent_uuid: "884a923cef7e4941bee4372c54754fe6" member_type: VOTER last_known_addr { host: "127.15.57.132" port: 35759 } } peers { permanent_uuid: "2f3297543b0540c88240093f71e19142" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 45465 } attrs { } }
I20260812 08:03:50.710343 20269 heartbeater.cc:507] Master 127.15.57.190:45861 requested a full tablet report, sending...
I20260812 08:03:50.710631 20077 ts_tablet_manager.cc:1434] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260812 08:03:50.717044 19480 ts_manager.cc:194] Registered new tserver with Master: b86a6b5ae577467eb35b046a0f5bb2ad (127.15.57.129:32929)
I20260812 08:03:50.718421 19480 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.15.57.129:42999
I20260812 08:03:50.725688 20269 heartbeater.cc:499] Master 127.15.57.190:45861 was elected leader, sending a full tablet report...
W20260812 08:03:50.739189 20111 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:50.739189 20121 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:50.739189 20113 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:50.739189 20120 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:50.745810 19377 meta_cache.cc:302] tablet bfc834b79d014cf19efba2554c569556: replica d37ee240a1664042b3d420265ada87f8 (127.15.57.131:39311) has failed: Network error: Client connection negotiation failed: client connection to 127.15.57.131:39311: connect: Connection refused (error 111)
W20260812 08:03:50.756251 19883 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44620: Illegal state: replica 884a923cef7e4941bee4372c54754fe6 is not leader of this config: current role FOLLOWER
W20260812 08:03:50.756251 19884 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44620: Illegal state: replica 884a923cef7e4941bee4372c54754fe6 is not leader of this config: current role FOLLOWER
W20260812 08:03:50.756251 19882 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44620: Illegal state: replica 884a923cef7e4941bee4372c54754fe6 is not leader of this config: current role FOLLOWER
W20260812 08:03:50.756251 19881 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44620: Illegal state: replica 884a923cef7e4941bee4372c54754fe6 is not leader of this config: current role FOLLOWER
W20260812 08:03:50.777570 20120 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:50.777554 20113 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:50.783573 19883 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44620: Illegal state: replica 884a923cef7e4941bee4372c54754fe6 is not leader of this config: current role FOLLOWER
W20260812 08:03:50.788944 20113 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:50.797355 20113 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:50.804800 20113 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:50.817320 20113 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:50.831653 20113 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:50.844681 20113 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:50.862493 20120 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:50.862526 20113 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:50.862526 20111 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:50.875800 19883 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44620: Illegal state: replica 884a923cef7e4941bee4372c54754fe6 is not leader of this config: current role FOLLOWER
W20260812 08:03:50.878294 19883 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44620: Illegal state: replica 884a923cef7e4941bee4372c54754fe6 is not leader of this config: current role FOLLOWER
W20260812 08:03:50.882318 20111 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:50.902186 20111 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:50.920387 20111 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:50.934000 20111 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:50.954439 20111 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:50.976961 20111 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.001624 20113 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.001624 20111 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.024652 20113 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.024873 20111 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.049715 20111 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.050101 20113 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.076061 20113 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.076061 20111 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.106546 20113 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.106546 20111 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.106678 20120 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.132099 19883 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44620: Illegal state: replica 884a923cef7e4941bee4372c54754fe6 is not leader of this config: current role FOLLOWER
W20260812 08:03:51.135066 19883 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44620: Illegal state: replica 884a923cef7e4941bee4372c54754fe6 is not leader of this config: current role FOLLOWER
W20260812 08:03:51.135308 19884 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44620: Illegal state: replica 884a923cef7e4941bee4372c54754fe6 is not leader of this config: current role FOLLOWER
W20260812 08:03:51.164285 20120 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.164469 20111 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.194518 20120 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.225680 20120 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.225855 20111 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.226315 20113 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.256415 19884 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44620: Illegal state: replica 884a923cef7e4941bee4372c54754fe6 is not leader of this config: current role FOLLOWER
W20260812 08:03:51.256415 19883 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44620: Illegal state: replica 884a923cef7e4941bee4372c54754fe6 is not leader of this config: current role FOLLOWER
W20260812 08:03:51.257447 19883 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44620: Illegal state: replica 884a923cef7e4941bee4372c54754fe6 is not leader of this config: current role FOLLOWER
I20260812 08:03:51.267853 19802 heartbeater.cc:499] Master 127.15.57.190:45861 was elected leader, sending a full tablet report...
W20260812 08:03:51.292644 20111 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.292644 20113 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.293314 20121 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.293319 20120 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.325783 19883 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44620: Illegal state: replica 884a923cef7e4941bee4372c54754fe6 is not leader of this config: current role FOLLOWER
W20260812 08:03:51.329420 19883 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44620: Illegal state: replica 884a923cef7e4941bee4372c54754fe6 is not leader of this config: current role FOLLOWER
W20260812 08:03:51.357070 19883 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44620: Illegal state: replica 884a923cef7e4941bee4372c54754fe6 is not leader of this config: current role FOLLOWER
W20260812 08:03:51.361603 20120 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.397017 20120 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.397476 20121 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.432976 20121 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.468565 20121 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.474308 19883 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44620: Illegal state: replica 884a923cef7e4941bee4372c54754fe6 is not leader of this config: current role FOLLOWER
W20260812 08:03:51.508654 20121 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.547931 20120 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.547931 20121 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.548556 20113 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.588753 19883 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44620: Illegal state: replica 884a923cef7e4941bee4372c54754fe6 is not leader of this config: current role FOLLOWER
W20260812 08:03:51.588753 19884 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44620: Illegal state: replica 884a923cef7e4941bee4372c54754fe6 is not leader of this config: current role FOLLOWER
W20260812 08:03:51.589974 19884 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44620: Illegal state: replica 884a923cef7e4941bee4372c54754fe6 is not leader of this config: current role FOLLOWER
W20260812 08:03:51.632138 20113 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.632440 20121 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
W20260812 08:03:51.673779 20121 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
I20260812 08:03:51.699565 20294 raft_consensus.cc:493] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:51.699888 20294 raft_consensus.cc:515] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 32929 } } peers { permanent_uuid: "d37ee240a1664042b3d420265ada87f8" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 39311 } } peers { permanent_uuid: "884a923cef7e4941bee4372c54754fe6" member_type: VOTER last_known_addr { host: "127.15.57.132" port: 35759 } } peers { permanent_uuid: "2f3297543b0540c88240093f71e19142" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 45465 } attrs { } }
I20260812 08:03:51.701449 20294 leader_election.cc:290] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b86a6b5ae577467eb35b046a0f5bb2ad (127.15.57.129:32929), d37ee240a1664042b3d420265ada87f8 (127.15.57.131:39311), 2f3297543b0540c88240093f71e19142 (127.15.57.130:45465)
W20260812 08:03:51.702495 19839 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.57.131:39311: connect: Connection refused (error 111)
W20260812 08:03:51.703572 19839 leader_election.cc:336] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d37ee240a1664042b3d420265ada87f8 (127.15.57.131:39311): Network error: Client connection negotiation failed: client connection to 127.15.57.131:39311: connect: Connection refused (error 111)
I20260812 08:03:51.706498 20193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc834b79d014cf19efba2554c569556" candidate_uuid: "884a923cef7e4941bee4372c54754fe6" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" is_pre_election: true
I20260812 08:03:51.706929 20193 raft_consensus.cc:2468] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884a923cef7e4941bee4372c54754fe6 in term 1.
I20260812 08:03:51.706498 19728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc834b79d014cf19efba2554c569556" candidate_uuid: "884a923cef7e4941bee4372c54754fe6" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2f3297543b0540c88240093f71e19142" is_pre_election: true
I20260812 08:03:51.707768 19728 raft_consensus.cc:1803] T bfc834b79d014cf19efba2554c569556 P 2f3297543b0540c88240093f71e19142 [term 1 FOLLOWER]: voting while tombstoned based on last-logged opid 1.0
I20260812 08:03:51.708222 19728 raft_consensus.cc:2468] T bfc834b79d014cf19efba2554c569556 P 2f3297543b0540c88240093f71e19142 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 884a923cef7e4941bee4372c54754fe6 in term 1.
I20260812 08:03:51.708712 19835 leader_election.cc:304] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2f3297543b0540c88240093f71e19142, 884a923cef7e4941bee4372c54754fe6, b86a6b5ae577467eb35b046a0f5bb2ad; no voters: d37ee240a1664042b3d420265ada87f8
I20260812 08:03:51.709070 20294 raft_consensus.cc:2804] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 08:03:51.709187 20294 raft_consensus.cc:493] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:03:51.709259 20294 raft_consensus.cc:3060] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:03:51.711306 20294 raft_consensus.cc:515] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 32929 } } peers { permanent_uuid: "d37ee240a1664042b3d420265ada87f8" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 39311 } } peers { permanent_uuid: "884a923cef7e4941bee4372c54754fe6" member_type: VOTER last_known_addr { host: "127.15.57.132" port: 35759 } } peers { permanent_uuid: "2f3297543b0540c88240093f71e19142" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 45465 } attrs { } }
I20260812 08:03:51.711898 20294 leader_election.cc:290] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 [CANDIDATE]: Term 2 election: Requested vote from peers b86a6b5ae577467eb35b046a0f5bb2ad (127.15.57.129:32929), d37ee240a1664042b3d420265ada87f8 (127.15.57.131:39311), 2f3297543b0540c88240093f71e19142 (127.15.57.130:45465)
I20260812 08:03:51.712343 20193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc834b79d014cf19efba2554c569556" candidate_uuid: "884a923cef7e4941bee4372c54754fe6" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad"
I20260812 08:03:51.712532 20193 raft_consensus.cc:3060] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:03:51.712697 19728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc834b79d014cf19efba2554c569556" candidate_uuid: "884a923cef7e4941bee4372c54754fe6" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2f3297543b0540c88240093f71e19142"
I20260812 08:03:51.712823 19728 raft_consensus.cc:1803] T bfc834b79d014cf19efba2554c569556 P 2f3297543b0540c88240093f71e19142 [term 1 FOLLOWER]: voting while tombstoned based on last-logged opid 1.0
I20260812 08:03:51.712913 19728 raft_consensus.cc:3060] T bfc834b79d014cf19efba2554c569556 P 2f3297543b0540c88240093f71e19142 [term 1 FOLLOWER]: Advancing to term 2
W20260812 08:03:51.713300 19839 leader_election.cc:336] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d37ee240a1664042b3d420265ada87f8 (127.15.57.131:39311): Network error: Client connection negotiation failed: client connection to 127.15.57.131:39311: connect: Connection refused (error 111)
I20260812 08:03:51.714270 19728 raft_consensus.cc:2468] T bfc834b79d014cf19efba2554c569556 P 2f3297543b0540c88240093f71e19142 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884a923cef7e4941bee4372c54754fe6 in term 2.
I20260812 08:03:51.714977 20193 raft_consensus.cc:2468] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 884a923cef7e4941bee4372c54754fe6 in term 2.
I20260812 08:03:51.715449 19836 leader_election.cc:304] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2f3297543b0540c88240093f71e19142, 884a923cef7e4941bee4372c54754fe6, b86a6b5ae577467eb35b046a0f5bb2ad; no voters: d37ee240a1664042b3d420265ada87f8
I20260812 08:03:51.715653 20294 raft_consensus.cc:2804] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 08:03:51.715970 20294 raft_consensus.cc:697] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 [term 2 LEADER]: Becoming Leader. State: Replica: 884a923cef7e4941bee4372c54754fe6, State: Running, Role: LEADER
W20260812 08:03:51.716486 20121 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
I20260812 08:03:51.716732 20294 consensus_queue.cc:237] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 32929 } } peers { permanent_uuid: "d37ee240a1664042b3d420265ada87f8" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 39311 } } peers { permanent_uuid: "884a923cef7e4941bee4372c54754fe6" member_type: VOTER last_known_addr { host: "127.15.57.132" port: 35759 } } peers { permanent_uuid: "2f3297543b0540c88240093f71e19142" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 45465 } attrs { } }
W20260812 08:03:51.720002 20121 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43588: Illegal state: replica b86a6b5ae577467eb35b046a0f5bb2ad is not leader of this config: current role FOLLOWER
I20260812 08:03:51.720832 19481 catalog_manager.cc:5719] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 reported cstate change: term changed from 1 to 2, leader changed from b86a6b5ae577467eb35b046a0f5bb2ad (127.15.57.129) to 884a923cef7e4941bee4372c54754fe6 (127.15.57.132). New cstate: current_term: 2 leader_uuid: "884a923cef7e4941bee4372c54754fe6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 32929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d37ee240a1664042b3d420265ada87f8" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 39311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "884a923cef7e4941bee4372c54754fe6" member_type: VOTER last_known_addr { host: "127.15.57.132" port: 35759 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f3297543b0540c88240093f71e19142" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 45465 } attrs { } health_report { overall_health: UNKNOWN } } }
W20260812 08:03:51.760082 19377 meta_cache.cc:302] tablet bfc834b79d014cf19efba2554c569556: replica d37ee240a1664042b3d420265ada87f8 (127.15.57.131:39311) has failed: Network error: Client connection negotiation failed: client connection to 127.15.57.131:39311: connect: Connection refused (error 111)
I20260812 08:03:51.768151 20193 raft_consensus.cc:1275] T bfc834b79d014cf19efba2554c569556 P b86a6b5ae577467eb35b046a0f5bb2ad [term 2 FOLLOWER]: Refusing update from remote peer 884a923cef7e4941bee4372c54754fe6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
W20260812 08:03:51.768247 19835 consensus_peers.cc:597] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 -> Peer 2f3297543b0540c88240093f71e19142 (127.15.57.130:45465): Couldn't send request to peer 2f3297543b0540c88240093f71e19142. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260812 08:03:51.768429 19839 consensus_peers.cc:597] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 -> Peer d37ee240a1664042b3d420265ada87f8 (127.15.57.131:39311): Couldn't send request to peer d37ee240a1664042b3d420265ada87f8. Status: Network error: Client connection negotiation failed: client connection to 127.15.57.131:39311: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 08:03:51.769100 20294 consensus_queue.cc:1048] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 08:03:52.301589 20323 tablet_replica.cc:333] stopping tablet replica
I20260812 08:03:52.302035 20323 raft_consensus.cc:2243] T bfc834b79d014cf19efba2554c569556 P 2f3297543b0540c88240093f71e19142 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260812 08:03:52.302228 20323 raft_consensus.cc:2272] T bfc834b79d014cf19efba2554c569556 P 2f3297543b0540c88240093f71e19142 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 08:03:52.302585 20323 ts_tablet_manager.cc:933] T bfc834b79d014cf19efba2554c569556 P 2f3297543b0540c88240093f71e19142: Initiating tablet copy from peer 884a923cef7e4941bee4372c54754fe6 (127.15.57.132:35759)
I20260812 08:03:52.303671 20323 tablet_copy_client.cc:323] T bfc834b79d014cf19efba2554c569556 P 2f3297543b0540c88240093f71e19142: tablet copy: Beginning tablet copy session from remote peer at address 127.15.57.132:35759
I20260812 08:03:52.309605 19990 tablet_copy_service.cc:140] P 884a923cef7e4941bee4372c54754fe6: Received BeginTabletCopySession request for tablet bfc834b79d014cf19efba2554c569556 from peer 2f3297543b0540c88240093f71e19142 ({username='slave'} at 127.15.57.130:40357)
I20260812 08:03:52.309813 19990 tablet_copy_service.cc:161] P 884a923cef7e4941bee4372c54754fe6: Beginning new tablet copy session on tablet bfc834b79d014cf19efba2554c569556 from peer 2f3297543b0540c88240093f71e19142 at {username='slave'} at 127.15.57.130:40357: session id = 2f3297543b0540c88240093f71e19142-bfc834b79d014cf19efba2554c569556
I20260812 08:03:52.311273 19990 tablet_copy_source_session.cc:215] T bfc834b79d014cf19efba2554c569556 P 884a923cef7e4941bee4372c54754fe6: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 08:03:52.313366 20323 ts_tablet_manager.cc:1916] T bfc834b79d014cf19efba2554c569556 P 2f3297543b0540c88240093f71e19142: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260812 08:03:52.315598 20323 ts_tablet_manager.cc:1929] T bfc834b79d014cf19efba2554c569556 P 2f3297543b0540c88240093f71e19142: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.0
I20260812 08:03:52.316082 20323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfc834b79d014cf19efba2554c569556. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:52.317488 20323 tablet_copy_client.cc:806] T bfc834b79d014cf19efba2554c569556 P 2f3297543b0540c88240093f71e19142: tablet copy: Starting download of 0 data blocks...
I20260812 08:03:52.317828 20323 tablet_copy_client.cc:670] T bfc834b79d014cf19efba2554c569556 P 2f3297543b0540c88240093f71e19142: tablet copy: Starting download of 1 WAL segments...
I20260812 08:03:52.320593 20323 tablet_copy_client.cc:538] T bfc834b79d014cf19efba2554c569556 P 2f3297543b0540c88240093f71e19142: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 08:03:52.322443 20323 tablet_bootstrap.cc:492] T bfc834b79d014cf19efba2554c569556 P 2f3297543b0540c88240093f71e19142: Bootstrap starting.
I20260812 08:03:52.329514 20323 log.cc:826] T bfc834b79d014cf19efba2554c569556 P 2f3297543b0540c88240093f71e19142: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:52.333674 20323 tablet_bootstrap.cc:492] T bfc834b79d014cf19efba2554c569556 P 2f3297543b0540c88240093f71e19142: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 5 replicates
I20260812 08:03:52.334137 20323 tablet_bootstrap.cc:492] T bfc834b79d014cf19efba2554c569556 P 2f3297543b0540c88240093f71e19142: Bootstrap complete.
I20260812 08:03:52.334509 20323 ts_tablet_manager.cc:1403] T bfc834b79d014cf19efba2554c569556 P 2f3297543b0540c88240093f71e19142: Time spent bootstrapping tablet: real 0.012s	user 0.010s	sys 0.000s
I20260812 08:03:52.337073 20323 raft_consensus.cc:359] T bfc834b79d014cf19efba2554c569556 P 2f3297543b0540c88240093f71e19142 [term 2 FOLLOWER]: Replica starting. Triggering 5 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 32929 } } peers { permanent_uuid: "d37ee240a1664042b3d420265ada87f8" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 39311 } } peers { permanent_uuid: "884a923cef7e4941bee4372c54754fe6" member_type: VOTER last_known_addr { host: "127.15.57.132" port: 35759 } } peers { permanent_uuid: "2f3297543b0540c88240093f71e19142" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 45465 } attrs { } }
I20260812 08:03:52.339737 20323 raft_consensus.cc:740] T bfc834b79d014cf19efba2554c569556 P 2f3297543b0540c88240093f71e19142 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f3297543b0540c88240093f71e19142, State: Initialized, Role: FOLLOWER
I20260812 08:03:52.340441 20323 consensus_queue.cc:260] T bfc834b79d014cf19efba2554c569556 P 2f3297543b0540c88240093f71e19142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b86a6b5ae577467eb35b046a0f5bb2ad" member_type: VOTER last_known_addr { host: "127.15.57.129" port: 32929 } } peers { permanent_uuid: "d37ee240a1664042b3d420265ada87f8" member_type: VOTER last_known_addr { host: "127.15.57.131" port: 39311 } } peers { permanent_uuid: "884a923cef7e4941bee4372c54754fe6" member_type: VOTER last_known_addr { host: "127.15.57.132" port: 35759 } } peers { permanent_uuid: "2f3297543b0540c88240093f71e19142" member_type: VOTER last_known_addr { host: "127.15.57.130" port: 45465 } attrs { } }
I20260812 08:03:52.341528 20323 ts_tablet_manager.cc:1434] T bfc834b79d014cf19efba2554c569556 P 2f3297543b0540c88240093f71e19142: Time spent starting tablet: real 0.007s	user 0.010s	sys 0.000s
I20260812 08:03:52.342764 19990 tablet_copy_service.cc:342] P 884a923cef7e4941bee4372c54754fe6: Request end of tablet copy session 2f3297543b0540c88240093f71e19142-bfc834b79d014cf19efba2554c569556 received from {username='slave'} at 127.15.57.130:40357
I20260812 08:03:52.342958 19990 tablet_copy_service.cc:434] P 884a923cef7e4941bee4372c54754fe6: ending tablet copy session 2f3297543b0540c88240093f71e19142-bfc834b79d014cf19efba2554c569556 on tablet bfc834b79d014cf19efba2554c569556 with peer 2f3297543b0540c88240093f71e19142
I20260812 08:03:52.839712 19728 raft_consensus.cc:1217] T bfc834b79d014cf19efba2554c569556 P 2f3297543b0540c88240093f71e19142 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.7]   Dedup: 2.7->[]
I20260812 08:03:52.843432 20330 mvcc.cc:204] Tried to move back new op lower bound from 7317593423291121664 to 7317593422716063744. Current Snapshot: MvccSnapshot[applied={T|T < 7317593422911930368}]
I20260812 08:03:52.846895 20339 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:44620 (ReqId={client: 3e65203c671640daa7adaf9311c1b081, seq_no=2, attempt_no=41}) took 1044 ms. Trace:
I20260812 08:03:52.847493 20338 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:44620 (ReqId={client: 3e65203c671640daa7adaf9311c1b081, seq_no=0, attempt_no=40}) took 1085 ms. Trace:
I20260812 08:03:52.847118 20339 rpcz_store.cc:276] 0812 08:03:51.802480 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:03:51.802621 (+   141us) service_pool.cc:224] Handling call
0812 08:03:52.846869 (+1044248us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:03:51.803522 (+     0us) write_op.cc:183] PREPARE: starting on tablet bfc834b79d014cf19efba2554c569556
0812 08:03:51.803696 (+   174us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:03:51.803703 (+     7us) write_op.cc:435] Acquired schema lock
0812 08:03:51.803706 (+     3us) tablet.cc:662] Decoding operations
0812 08:03:51.803995 (+   289us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:03:51.804035 (+    40us) write_op.cc:641] Partition lock acquired for write op
0812 08:03:51.804039 (+     4us) tablet.cc:685] Acquiring locks for 50 operations
0812 08:03:51.804236 (+   197us) tablet.cc:701] Row locks acquired
0812 08:03:51.804238 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:03:51.804282 (+    44us) write_op.cc:270] Start()
0812 08:03:51.804366 (+    84us) write_op.cc:276] Timestamp: P: 1786521831804212 usec, L: 0
0812 08:03:51.804367 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 08:03:51.804564 (+   197us) log.cc:844] Serialized 2455 byte log entry
0812 08:03:52.842160 (+1037596us) op_driver.cc:464] REPLICATION: finished
0812 08:03:52.843257 (+  1097us) write_op.cc:301] APPLY: starting
0812 08:03:52.843284 (+    27us) tablet.cc:1367] starting BulkCheckPresence
0812 08:03:52.843399 (+   115us) tablet.cc:1370] finished BulkCheckPresence
0812 08:03:52.843420 (+    21us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:03:52.845801 (+  2381us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:03:52.845976 (+   175us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:03:52.845997 (+    21us) write_op.cc:312] APPLY: finished
0812 08:03:52.846324 (+   327us) log.cc:844] Serialized 319 byte log entry
0812 08:03:52.846505 (+   181us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:03:52.846592 (+    87us) write_op.cc:454] Released schema lock
0812 08:03:52.846618 (+    26us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1034,"num_ops":50,"prepare.queue_time_us":63,"prepare.run_cpu_time_us":1236,"prepare.run_wall_time_us":1242,"raft.queue_time_us":510,"raft.run_cpu_time_us":558,"raft.run_wall_time_us":807,"replication_time_us":1037775,"spinlock_wait_cycles":2560,"thread_start_us":62,"threads_started":1}]]}
I20260812 08:03:52.847741 20338 rpcz_store.cc:276] 0812 08:03:51.762403 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:03:51.762521 (+   118us) service_pool.cc:224] Handling call
0812 08:03:52.847465 (+1084944us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:03:51.763597 (+     0us) write_op.cc:183] PREPARE: starting on tablet bfc834b79d014cf19efba2554c569556
0812 08:03:51.763778 (+   181us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:03:51.763790 (+    12us) write_op.cc:435] Acquired schema lock
0812 08:03:51.763792 (+     2us) tablet.cc:662] Decoding operations
0812 08:03:51.764563 (+   771us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:03:51.764635 (+    72us) write_op.cc:641] Partition lock acquired for write op
0812 08:03:51.764641 (+     6us) tablet.cc:685] Acquiring locks for 50 operations
0812 08:03:51.765588 (+   947us) tablet.cc:701] Row locks acquired
0812 08:03:51.765590 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:03:51.765680 (+    90us) write_op.cc:270] Start()
0812 08:03:51.765872 (+   192us) write_op.cc:276] Timestamp: P: 1786521831765608 usec, L: 0
0812 08:03:51.765876 (+     4us) op_driver.cc:348] REPLICATION: starting
0812 08:03:51.766240 (+   364us) log.cc:844] Serialized 2455 byte log entry
0812 08:03:52.841973 (+1075733us) op_driver.cc:464] REPLICATION: finished
0812 08:03:52.842363 (+   390us) write_op.cc:301] APPLY: starting
0812 08:03:52.842514 (+   151us) tablet.cc:1367] starting BulkCheckPresence
0812 08:03:52.843266 (+   752us) tablet.cc:1370] finished BulkCheckPresence
0812 08:03:52.843320 (+    54us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:03:52.845803 (+  2483us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:03:52.846886 (+  1083us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:03:52.846893 (+     7us) write_op.cc:312] APPLY: finished
0812 08:03:52.847107 (+   214us) log.cc:844] Serialized 319 byte log entry
0812 08:03:52.847256 (+   149us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:03:52.847288 (+    32us) write_op.cc:454] Released schema lock
0812 08:03:52.847300 (+    12us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":247,"mutex_wait_us":262,"num_ops":50,"prepare.queue_time_us":361,"prepare.run_cpu_time_us":3078,"prepare.run_wall_time_us":3088,"raft.queue_time_us":478,"raft.run_cpu_time_us":968,"raft.run_wall_time_us":968,"replication_time_us":1076022,"spinlock_wait_cycles":124928,"thread_start_us":448,"threads_started":3}]]}
I20260812 08:03:52.851192 20338 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:44620 (ReqId={client: 3e65203c671640daa7adaf9311c1b081, seq_no=3, attempt_no=41}) took 1047 ms. Trace:
I20260812 08:03:52.852010 20338 rpcz_store.cc:276] 0812 08:03:51.803472 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 08:03:51.803535 (+    63us) service_pool.cc:224] Handling call
0812 08:03:52.851175 (+1047640us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 08:03:51.804771 (+     0us) write_op.cc:183] PREPARE: starting on tablet bfc834b79d014cf19efba2554c569556
0812 08:03:51.804892 (+   121us) write_op.cc:432] Acquiring schema lock in shared mode
0812 08:03:51.804897 (+     5us) write_op.cc:435] Acquired schema lock
0812 08:03:51.804898 (+     1us) tablet.cc:662] Decoding operations
0812 08:03:51.805148 (+   250us) write_op.cc:620] Acquiring the partition lock for write op
0812 08:03:51.805158 (+    10us) write_op.cc:641] Partition lock acquired for write op
0812 08:03:51.805162 (+     4us) tablet.cc:685] Acquiring locks for 50 operations
0812 08:03:51.805395 (+   233us) tablet.cc:701] Row locks acquired
0812 08:03:51.805397 (+     2us) write_op.cc:260] PREPARE: finished
0812 08:03:51.805442 (+    45us) write_op.cc:270] Start()
0812 08:03:51.805506 (+    64us) write_op.cc:276] Timestamp: P: 1786521831805372 usec, L: 0
0812 08:03:51.805508 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 08:03:51.805665 (+   157us) log.cc:844] Serialized 2455 byte log entry
0812 08:03:52.842291 (+1036626us) op_driver.cc:464] REPLICATION: finished
0812 08:03:52.849268 (+  6977us) write_op.cc:301] APPLY: starting
0812 08:03:52.849309 (+    41us) tablet.cc:1367] starting BulkCheckPresence
0812 08:03:52.849452 (+   143us) tablet.cc:1370] finished BulkCheckPresence
0812 08:03:52.849493 (+    41us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 08:03:52.850538 (+  1045us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 08:03:52.850618 (+    80us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 08:03:52.850624 (+     6us) write_op.cc:312] APPLY: finished
0812 08:03:52.850837 (+   213us) log.cc:844] Serialized 319 byte log entry
0812 08:03:52.850943 (+   106us) write_op.cc:489] Releasing partition, row and schema locks
0812 08:03:52.850974 (+    31us) write_op.cc:454] Released schema lock
0812 08:03:52.851023 (+    49us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":6936,"num_ops":50,"prepare.queue_time_us":906,"prepare.run_cpu_time_us":1037,"prepare.run_wall_time_us":1037,"raft.queue_time_us":147,"raft.run_cpu_time_us":7,"raft.run_wall_time_us":8,"replication_time_us":1036769,"spinlock_wait_cycles":110080,"thread_start_us":98,"threads_started":1}]]}
I20260812 08:03:52.858403 20352 mvcc.cc:204] Tried to move back new op lower bound from 7317593423291121664 to 7317593422716063744. Current Snapshot: MvccSnapshot[applied={T|T < 7317593422911930368}]
I20260812 08:03:53.731285 15590 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu with pid 20042
I20260812 08:03:53.749702 15590 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu with pid 19551
I20260812 08:03:53.778689 15590 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu with pid 19807
I20260812 08:03:53.808568 15590 external_mini_cluster.cc:1664] Killing /tmp/dist-test-tasklDZR8P/build/debug/bin/kudu with pid 19437
2026-08-12T08:03:53Z chronyd exiting
[       OK ] FailureCause/TabletCopyFailureITest.TestTabletCopyNewReplicaFailureCanVote/1 (7037 ms)
[----------] 1 test from FailureCause/TabletCopyFailureITest (7037 ms total)

[----------] Global test environment tear-down
[==========] 4 tests from 2 test suites ran. (20716 ms total)
[  PASSED  ] 4 tests.
I20260812 08:03:53.849622 15590 logging.cc:436] LogThrottler /workspace/apache/dev/local/kudu/src/kudu/client/meta_cache.cc:302: suppressed but not reported on 2 messages since previous log ~2 seconds ago
