[==========] Running 6 tests from 3 test suites.
[----------] Global test environment set-up.
[----------] 3 tests from PrometheusSDITest
[ RUN      ] PrometheusSDITest.BasicSDAndScrape
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260812 06:36:37.799434  1286 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.1.65.190:33491
I20260812 06:36:37.800336  1286 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:36:37.800933  1286 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:36:37.809270  1296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:37.810945  1294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:37.811273  1298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:37.811575  1286 server_base.cc:1061] running on GCE node
I20260812 06:36:37.812183  1286 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 06:36:37.812319  1286 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 06:36:37.812371  1286 hybrid_clock.cc:648] HybridClock initialized: now 1786516597812368 us; error 0 us; skew 500 ppm
I20260812 06:36:37.814271  1286 webserver.cc:533] Webserver started at http://127.1.65.190:36065/ using document root <none> and password file <none>
I20260812 06:36:37.814867  1286 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:37.814958  1286 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:37.815300  1286 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:36:37.816953  1286 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.BasicSDAndScrape.1786516597789727-1286-0/minicluster-data/master-0-root/instance:
uuid: "c8360d1191a3431eb92a100ae54f0f55"
format_stamp: "Formatted at 2026-08-12 06:36:37 on dist-test-slave-92m1"
I20260812 06:36:37.820813  1286 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 06:36:37.822834  1306 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:37.823786  1286 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:36:37.823947  1286 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.BasicSDAndScrape.1786516597789727-1286-0/minicluster-data/master-0-root
uuid: "c8360d1191a3431eb92a100ae54f0f55"
format_stamp: "Formatted at 2026-08-12 06:36:37 on dist-test-slave-92m1"
I20260812 06:36:37.824052  1286 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.BasicSDAndScrape.1786516597789727-1286-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.BasicSDAndScrape.1786516597789727-1286-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.BasicSDAndScrape.1786516597789727-1286-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:37.857520  1286 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:37.858258  1286 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:36:37.858443  1286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:37.867621  1286 rpc_server.cc:307] RPC server started. Bound to: 127.1.65.190:33491
I20260812 06:36:37.867636  1394 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.65.190:33491 every 8 connection(s)
I20260812 06:36:37.870916  1395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:36:37.876616  1395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8360d1191a3431eb92a100ae54f0f55: Bootstrap starting.
I20260812 06:36:37.879056  1395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8360d1191a3431eb92a100ae54f0f55: Neither blocks nor log segments found. Creating new log.
I20260812 06:36:37.880012  1395 log.cc:826] T 00000000000000000000000000000000 P c8360d1191a3431eb92a100ae54f0f55: Log is configured to *not* fsync() on all Append() calls
I20260812 06:36:37.881536  1395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8360d1191a3431eb92a100ae54f0f55: No bootstrap required, opened a new log
I20260812 06:36:37.884483  1395 raft_consensus.cc:359] T 00000000000000000000000000000000 P c8360d1191a3431eb92a100ae54f0f55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8360d1191a3431eb92a100ae54f0f55" member_type: VOTER }
I20260812 06:36:37.884646  1395 raft_consensus.cc:385] T 00000000000000000000000000000000 P c8360d1191a3431eb92a100ae54f0f55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:36:37.884730  1395 raft_consensus.cc:740] T 00000000000000000000000000000000 P c8360d1191a3431eb92a100ae54f0f55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8360d1191a3431eb92a100ae54f0f55, State: Initialized, Role: FOLLOWER
I20260812 06:36:37.885313  1395 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8360d1191a3431eb92a100ae54f0f55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8360d1191a3431eb92a100ae54f0f55" member_type: VOTER }
I20260812 06:36:37.885450  1395 raft_consensus.cc:399] T 00000000000000000000000000000000 P c8360d1191a3431eb92a100ae54f0f55 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:36:37.885505  1395 raft_consensus.cc:493] T 00000000000000000000000000000000 P c8360d1191a3431eb92a100ae54f0f55 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:36:37.885592  1395 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c8360d1191a3431eb92a100ae54f0f55 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:36:37.886256  1395 raft_consensus.cc:515] T 00000000000000000000000000000000 P c8360d1191a3431eb92a100ae54f0f55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8360d1191a3431eb92a100ae54f0f55" member_type: VOTER }
I20260812 06:36:37.886626  1395 leader_election.cc:304] T 00000000000000000000000000000000 P c8360d1191a3431eb92a100ae54f0f55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8360d1191a3431eb92a100ae54f0f55; no voters: 
I20260812 06:36:37.886875  1395 leader_election.cc:290] T 00000000000000000000000000000000 P c8360d1191a3431eb92a100ae54f0f55 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 06:36:37.887045  1401 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c8360d1191a3431eb92a100ae54f0f55 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:36:37.887475  1401 raft_consensus.cc:697] T 00000000000000000000000000000000 P c8360d1191a3431eb92a100ae54f0f55 [term 1 LEADER]: Becoming Leader. State: Replica: c8360d1191a3431eb92a100ae54f0f55, State: Running, Role: LEADER
I20260812 06:36:37.887827  1401 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8360d1191a3431eb92a100ae54f0f55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8360d1191a3431eb92a100ae54f0f55" member_type: VOTER }
I20260812 06:36:37.887969  1395 sys_catalog.cc:565] T 00000000000000000000000000000000 P c8360d1191a3431eb92a100ae54f0f55 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:36:37.890141  1403 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8360d1191a3431eb92a100ae54f0f55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8360d1191a3431eb92a100ae54f0f55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8360d1191a3431eb92a100ae54f0f55" member_type: VOTER } }
I20260812 06:36:37.890272  1403 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8360d1191a3431eb92a100ae54f0f55 [sys.catalog]: This master's current role is: LEADER
I20260812 06:36:37.890609  1404 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8360d1191a3431eb92a100ae54f0f55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8360d1191a3431eb92a100ae54f0f55. Latest consensus state: current_term: 1 leader_uuid: "c8360d1191a3431eb92a100ae54f0f55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8360d1191a3431eb92a100ae54f0f55" member_type: VOTER } }
I20260812 06:36:37.890661  1286 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260812 06:36:37.890692  1404 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8360d1191a3431eb92a100ae54f0f55 [sys.catalog]: This master's current role is: LEADER
I20260812 06:36:37.890671  1425 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:36:37.893347  1425 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:36:37.898164  1425 catalog_manager.cc:1383] Generated new cluster ID: 039b8e96f8d7408f8d4f8e9fd0663090
I20260812 06:36:37.898229  1425 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:36:37.907236  1425 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 06:36:37.908300  1425 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:36:37.915027  1425 catalog_manager.cc:6092] T 00000000000000000000000000000000 P c8360d1191a3431eb92a100ae54f0f55: Generated new TSK 0
I20260812 06:36:37.915704  1425 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 06:36:37.923555  1286 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:36:37.926522  1437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:37.926601  1433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:37.926674  1286 server_base.cc:1061] running on GCE node
W20260812 06:36:37.926594  1432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:37.927003  1286 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 06:36:37.927060  1286 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 06:36:37.927083  1286 hybrid_clock.cc:648] HybridClock initialized: now 1786516597927083 us; error 0 us; skew 500 ppm
I20260812 06:36:37.928069  1286 webserver.cc:533] Webserver started at http://127.1.65.129:44167/ using document root <none> and password file <none>
I20260812 06:36:37.928231  1286 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:37.928287  1286 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:37.928360  1286 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:36:37.928768  1286 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.BasicSDAndScrape.1786516597789727-1286-0/minicluster-data/ts-0-root/instance:
uuid: "2138dabab0074ca4a5a5f63b66019103"
format_stamp: "Formatted at 2026-08-12 06:36:37 on dist-test-slave-92m1"
I20260812 06:36:37.930619  1286 fs_manager.cc:696] Time spent creating directory manager: real 0.002s	user 0.001s	sys 0.002s
I20260812 06:36:37.931718  1443 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:37.932029  1286 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.000s
I20260812 06:36:37.932096  1286 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.BasicSDAndScrape.1786516597789727-1286-0/minicluster-data/ts-0-root
uuid: "2138dabab0074ca4a5a5f63b66019103"
format_stamp: "Formatted at 2026-08-12 06:36:37 on dist-test-slave-92m1"
I20260812 06:36:37.932178  1286 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.BasicSDAndScrape.1786516597789727-1286-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.BasicSDAndScrape.1786516597789727-1286-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.BasicSDAndScrape.1786516597789727-1286-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:37.978322  1286 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:37.978842  1286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:37.979480  1286 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:36:37.980499  1286 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:36:37.980557  1286 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:37.980630  1286 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:36:37.980675  1286 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:37.989806  1286 rpc_server.cc:307] RPC server started. Bound to: 127.1.65.129:33171
I20260812 06:36:37.989830  1560 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.65.129:33171 every 8 connection(s)
I20260812 06:36:37.999887  1561 heartbeater.cc:344] Connected to a master server at 127.1.65.190:33491
I20260812 06:36:38.000142  1561 heartbeater.cc:461] Registering TS with master...
I20260812 06:36:38.000608  1561 heartbeater.cc:507] Master 127.1.65.190:33491 requested a full tablet report, sending...
I20260812 06:36:38.002146  1333 ts_manager.cc:194] Registered new tserver with Master: 2138dabab0074ca4a5a5f63b66019103 (127.1.65.129:33171)
I20260812 06:36:38.002255  1286 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.011515752s
I20260812 06:36:38.002897  1286 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 4.9922e-05s
I20260812 06:36:38.003693  1333 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50592
time=2026-08-12T06:36:38.205Z level=INFO source=main.go:1678 msg="updated GOGC" old=100 new=75
time=2026-08-12T06:36:38.207Z level=INFO source=main.go:744 msg="Leaving GOMAXPROCS=4: CPU quota undefined" component=automaxprocs
time=2026-08-12T06:36:38.208Z level=INFO source=memlimit.go:198 msg="GOMEMLIMIT is updated" component=automemlimit package=github.com/KimMachineGun/automemlimit/memlimit GOMEMLIMIT=14142552883 previous=9223372036854775807
time=2026-08-12T06:36:38.209Z level=INFO source=main.go:851 msg="Starting Prometheus Server" mode=server version="(version=3.11.2, branch=HEAD, revision=f0f0fdd679dcd6df320b0558b20919f7cd44c407)"
time=2026-08-12T06:36:38.209Z level=INFO source=main.go:856 msg="operational information" build_context="(go=go1.26.2, platform=linux/amd64, user=root@d2684485c347, date=20260413-11:53:40, tags=netgo,builtinassets)" host_details="(Linux 5.4.0-1157-gcp #166~18.04.1-Ubuntu SMP Wed Jan 21 01:57:46 UTC 2026 x86_64 dist-test-slave-92m1 (none))" fd_limits="(soft=1048576, hard=1048576)" vm_limits="(soft=unlimited, hard=unlimited)"
time=2026-08-12T06:36:38.457Z level=INFO source=web.go:710 msg="Start listening for connections" component=web address=127.0.0.1:0
time=2026-08-12T06:36:38.458Z level=INFO source=main.go:1410 msg="Starting TSDB ..."
time=2026-08-12T06:36:38.462Z level=INFO source=tls_config.go:354 msg="Listening on" component=web address=127.0.0.1:39643
time=2026-08-12T06:36:38.462Z level=INFO source=tls_config.go:357 msg="TLS is disabled." component=web http2=false address=127.0.0.1:39643
time=2026-08-12T06:36:38.464Z level=INFO source=head.go:698 msg="Replaying on-disk memory mappable chunks if any" component=tsdb
time=2026-08-12T06:36:38.465Z level=INFO source=head.go:784 msg="On-disk memory mappable chunks replay completed" component=tsdb duration=69.832µs
time=2026-08-12T06:36:38.465Z level=INFO source=head.go:792 msg="Replaying WAL, this may take a while" component=tsdb
time=2026-08-12T06:36:38.466Z level=INFO source=head.go:865 msg="WAL segment loaded" component=tsdb segment=0 maxSegment=0 duration=826.162µs
time=2026-08-12T06:36:38.466Z level=INFO source=head.go:902 msg="WAL replay completed" component=tsdb checkpoint_replay_duration=93.632µs wal_replay_duration=909.885µs wbl_replay_duration=64ns chunk_snapshot_load_duration=0s mmap_chunk_replay_duration=69.832µs total_replay_duration=1.144599ms
time=2026-08-12T06:36:38.470Z level=INFO source=main.go:1431 msg="filesystem information" fs_type=OVERLAYFS_SUPER_MAGIC
time=2026-08-12T06:36:38.470Z level=INFO source=main.go:1434 msg="TSDB started"
time=2026-08-12T06:36:38.470Z level=INFO source=main.go:1632 msg="Loading configuration file" filename=/tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.BasicSDAndScrape.1786516597789727-1286-0/prometheus.yml
time=2026-08-12T06:36:38.471Z level=INFO source=main.go:1048 msg="TSDB retention updated" duration=15d size=0B percentage=0
time=2026-08-12T06:36:38.479Z level=INFO source=main.go:1671 msg="Completed loading of configuration file" db_storage=125.057µs remote_storage=21.297µs web_handler=720ns query_engine=1.655µs scrape=7.437706ms scrape_sd=406.356µs notify=20.205µs notify_sd=1.042µs rules=27.303µs tracing=179.287µs filename=/tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.BasicSDAndScrape.1786516597789727-1286-0/prometheus.yml totalDuration=8.942354ms
time=2026-08-12T06:36:38.479Z level=INFO source=main.go:1395 msg="Server is ready to receive web requests."
time=2026-08-12T06:36:38.479Z level=INFO source=manager.go:209 msg="Starting rule manager..." component="rule manager"
I20260812 06:36:39.011118  1561 heartbeater.cc:499] Master 127.1.65.190:33491 was elected leader, sending a full tablet report...
time=2026-08-12T06:36:45.433Z level=WARN source=main.go:1191 msg="Received an OS signal, exiting gracefully..." signal=terminated
time=2026-08-12T06:36:45.433Z level=INFO source=main.go:1216 msg="Stopping scrape discovery manager..."
time=2026-08-12T06:36:45.433Z level=INFO source=main.go:1230 msg="Stopping notify discovery manager..."
time=2026-08-12T06:36:45.433Z level=INFO source=main.go:1245 msg="Stopping rule manager manager..."
time=2026-08-12T06:36:45.433Z level=INFO source=main.go:1212 msg="Scrape discovery manager stopped"
time=2026-08-12T06:36:45.433Z level=INFO source=manager.go:223 msg="Stopping rule manager..." component="rule manager"
time=2026-08-12T06:36:45.433Z level=INFO source=manager.go:239 msg="Rule manager stopped" component="rule manager"
time=2026-08-12T06:36:45.433Z level=INFO source=main.go:1269 msg="Stopping scrape manager..."
time=2026-08-12T06:36:45.433Z level=INFO source=main.go:1226 msg="Notify discovery manager stopped"
time=2026-08-12T06:36:45.433Z level=INFO source=main.go:1284 msg="Stopping tracing manager..."
time=2026-08-12T06:36:45.433Z level=INFO source=main.go:1241 msg="Rule manager stopped"
time=2026-08-12T06:36:45.433Z level=INFO source=main.go:1369 msg="Stopping reloaders..."
time=2026-08-12T06:36:45.433Z level=INFO source=main.go:1261 msg="Scrape manager stopped"
time=2026-08-12T06:36:45.433Z level=INFO source=main.go:1458 msg="Stopping storage..."
time=2026-08-12T06:36:45.433Z level=INFO source=main.go:1280 msg="Tracing manager stopped"
time=2026-08-12T06:36:45.433Z level=INFO source=main.go:1361 msg="Reloaders stopped"
time=2026-08-12T06:36:45.438Z level=INFO source=main.go:1536 msg="Stopping web handler..."
time=2026-08-12T06:36:45.438Z level=INFO source=main.go:1559 msg="Stopping notifier manager..."
time=2026-08-12T06:36:45.438Z level=INFO source=main.go:1454 msg="TSDB stopped"
time=2026-08-12T06:36:45.438Z level=INFO source=manager.go:321 msg="Stopping notification manager..." component=notifier
time=2026-08-12T06:36:45.438Z level=INFO source=main.go:1555 msg="Notifier manager stopped"
time=2026-08-12T06:36:45.438Z level=INFO source=main.go:1532 msg="Web handler stopped"
time=2026-08-12T06:36:45.438Z level=INFO source=main.go:1570 msg="See you next time!"
I20260812 06:36:45.453750  1286 tablet_server.cc:179] TabletServer@127.1.65.129:0 shutting down...
I20260812 06:36:45.458461  1286 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260812 06:36:45.470995  1286 tablet_server.cc:196] TabletServer@127.1.65.129:0 shutdown complete.
I20260812 06:36:45.474494  1286 master.cc:562] Master@127.1.65.190:33491 shutting down...
I20260812 06:36:45.478753  1286 raft_consensus.cc:2243] T 00000000000000000000000000000000 P c8360d1191a3431eb92a100ae54f0f55 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:36:45.478964  1286 raft_consensus.cc:2272] T 00000000000000000000000000000000 P c8360d1191a3431eb92a100ae54f0f55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:36:45.479030  1286 tablet_replica.cc:333] T 00000000000000000000000000000000 P c8360d1191a3431eb92a100ae54f0f55: stopping tablet replica
I20260812 06:36:45.491966  1286 master.cc:584] Master@127.1.65.190:33491 shutdown complete.
[       OK ] PrometheusSDITest.BasicSDAndScrape (7698 ms)
[ RUN      ] PrometheusSDITest.SDLabels
I20260812 06:36:45.497709  1286 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.1.65.190:37649
I20260812 06:36:45.497999  1286 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:36:45.500022  1599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:45.500087  1601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:45.500171  1286 server_base.cc:1061] running on GCE node
W20260812 06:36:45.500377  1598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:45.500669  1286 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 06:36:45.500737  1286 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 06:36:45.500762  1286 hybrid_clock.cc:648] HybridClock initialized: now 1786516605500761 us; error 0 us; skew 500 ppm
I20260812 06:36:45.501597  1286 webserver.cc:533] Webserver started at http://127.1.65.190:36713/ using document root <none> and password file <none>
I20260812 06:36:45.501788  1286 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:45.501861  1286 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:45.501940  1286 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:36:45.502333  1286 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.SDLabels.1786516597789727-1286-0/minicluster-data/master-0-root/instance:
uuid: "38425d33d54a44e8bdde134246ab1cb1"
format_stamp: "Formatted at 2026-08-12 06:36:45 on dist-test-slave-92m1"
I20260812 06:36:45.503955  1286 fs_manager.cc:696] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20260812 06:36:45.504902  1609 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:45.505143  1286 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20260812 06:36:45.505239  1286 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.SDLabels.1786516597789727-1286-0/minicluster-data/master-0-root
uuid: "38425d33d54a44e8bdde134246ab1cb1"
format_stamp: "Formatted at 2026-08-12 06:36:45 on dist-test-slave-92m1"
I20260812 06:36:45.505330  1286 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.SDLabels.1786516597789727-1286-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.SDLabels.1786516597789727-1286-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.SDLabels.1786516597789727-1286-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:45.510615  1286 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:45.510950  1286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:45.515780  1286 rpc_server.cc:307] RPC server started. Bound to: 127.1.65.190:37649
I20260812 06:36:45.517812  1703 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.65.190:37649 every 8 connection(s)
I20260812 06:36:45.518258  1704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:36:45.520012  1704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38425d33d54a44e8bdde134246ab1cb1: Bootstrap starting.
I20260812 06:36:45.520790  1704 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38425d33d54a44e8bdde134246ab1cb1: Neither blocks nor log segments found. Creating new log.
I20260812 06:36:45.521746  1704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38425d33d54a44e8bdde134246ab1cb1: No bootstrap required, opened a new log
I20260812 06:36:45.522189  1704 raft_consensus.cc:359] T 00000000000000000000000000000000 P 38425d33d54a44e8bdde134246ab1cb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38425d33d54a44e8bdde134246ab1cb1" member_type: VOTER }
I20260812 06:36:45.522276  1704 raft_consensus.cc:385] T 00000000000000000000000000000000 P 38425d33d54a44e8bdde134246ab1cb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:36:45.522298  1704 raft_consensus.cc:740] T 00000000000000000000000000000000 P 38425d33d54a44e8bdde134246ab1cb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38425d33d54a44e8bdde134246ab1cb1, State: Initialized, Role: FOLLOWER
I20260812 06:36:45.522447  1704 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38425d33d54a44e8bdde134246ab1cb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38425d33d54a44e8bdde134246ab1cb1" member_type: VOTER }
I20260812 06:36:45.522548  1704 raft_consensus.cc:399] T 00000000000000000000000000000000 P 38425d33d54a44e8bdde134246ab1cb1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:36:45.522574  1704 raft_consensus.cc:493] T 00000000000000000000000000000000 P 38425d33d54a44e8bdde134246ab1cb1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:36:45.522605  1704 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 38425d33d54a44e8bdde134246ab1cb1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:36:45.523286  1704 raft_consensus.cc:515] T 00000000000000000000000000000000 P 38425d33d54a44e8bdde134246ab1cb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38425d33d54a44e8bdde134246ab1cb1" member_type: VOTER }
I20260812 06:36:45.523398  1704 leader_election.cc:304] T 00000000000000000000000000000000 P 38425d33d54a44e8bdde134246ab1cb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38425d33d54a44e8bdde134246ab1cb1; no voters: 
I20260812 06:36:45.523535  1704 leader_election.cc:290] T 00000000000000000000000000000000 P 38425d33d54a44e8bdde134246ab1cb1 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 06:36:45.523701  1712 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 38425d33d54a44e8bdde134246ab1cb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:36:45.523938  1712 raft_consensus.cc:697] T 00000000000000000000000000000000 P 38425d33d54a44e8bdde134246ab1cb1 [term 1 LEADER]: Becoming Leader. State: Replica: 38425d33d54a44e8bdde134246ab1cb1, State: Running, Role: LEADER
I20260812 06:36:45.524042  1704 sys_catalog.cc:565] T 00000000000000000000000000000000 P 38425d33d54a44e8bdde134246ab1cb1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:36:45.524092  1712 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38425d33d54a44e8bdde134246ab1cb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38425d33d54a44e8bdde134246ab1cb1" member_type: VOTER }
I20260812 06:36:45.524572  1713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38425d33d54a44e8bdde134246ab1cb1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38425d33d54a44e8bdde134246ab1cb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38425d33d54a44e8bdde134246ab1cb1" member_type: VOTER } }
I20260812 06:36:45.524670  1713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38425d33d54a44e8bdde134246ab1cb1 [sys.catalog]: This master's current role is: LEADER
I20260812 06:36:45.524632  1716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38425d33d54a44e8bdde134246ab1cb1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38425d33d54a44e8bdde134246ab1cb1. Latest consensus state: current_term: 1 leader_uuid: "38425d33d54a44e8bdde134246ab1cb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38425d33d54a44e8bdde134246ab1cb1" member_type: VOTER } }
I20260812 06:36:45.524750  1716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38425d33d54a44e8bdde134246ab1cb1 [sys.catalog]: This master's current role is: LEADER
I20260812 06:36:45.525003  1725 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:36:45.526238  1725 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:36:45.526432  1286 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260812 06:36:45.528201  1725 catalog_manager.cc:1383] Generated new cluster ID: 10a333a317804287b8762f30b519c88f
I20260812 06:36:45.528249  1725 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:36:45.538807  1725 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 06:36:45.539441  1725 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:36:45.554667  1725 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 38425d33d54a44e8bdde134246ab1cb1: Generated new TSK 0
I20260812 06:36:45.554896  1725 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 06:36:45.558722  1286 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:36:45.560923  1746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:45.561025  1751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:45.561159  1286 server_base.cc:1061] running on GCE node
W20260812 06:36:45.561045  1747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:45.561457  1286 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 06:36:45.561504  1286 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 06:36:45.561520  1286 hybrid_clock.cc:648] HybridClock initialized: now 1786516605561521 us; error 0 us; skew 500 ppm
I20260812 06:36:45.562345  1286 webserver.cc:533] Webserver started at http://127.1.65.129:37693/ using document root <none> and password file <none>
I20260812 06:36:45.562474  1286 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:45.562515  1286 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:45.562564  1286 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:36:45.562901  1286 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.SDLabels.1786516597789727-1286-0/minicluster-data/ts-0-root/instance:
uuid: "7dcdd15d85ac4c96b63e180ba0210c67"
format_stamp: "Formatted at 2026-08-12 06:36:45 on dist-test-slave-92m1"
I20260812 06:36:45.564549  1286 fs_manager.cc:696] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20260812 06:36:45.565549  1758 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:45.565923  1286 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20260812 06:36:45.566004  1286 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.SDLabels.1786516597789727-1286-0/minicluster-data/ts-0-root
uuid: "7dcdd15d85ac4c96b63e180ba0210c67"
format_stamp: "Formatted at 2026-08-12 06:36:45 on dist-test-slave-92m1"
I20260812 06:36:45.566107  1286 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.SDLabels.1786516597789727-1286-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.SDLabels.1786516597789727-1286-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.SDLabels.1786516597789727-1286-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:45.571916  1286 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:45.572294  1286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:45.572600  1286 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:36:45.573089  1286 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:36:45.573127  1286 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:45.573190  1286 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:36:45.573230  1286 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:45.578027  1286 rpc_server.cc:307] RPC server started. Bound to: 127.1.65.129:35381
I20260812 06:36:45.578053  1864 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.65.129:35381 every 8 connection(s)
I20260812 06:36:45.586619  1865 heartbeater.cc:344] Connected to a master server at 127.1.65.190:37649
I20260812 06:36:45.586725  1865 heartbeater.cc:461] Registering TS with master...
I20260812 06:36:45.586905  1865 heartbeater.cc:507] Master 127.1.65.190:37649 requested a full tablet report, sending...
I20260812 06:36:45.587594  1638 ts_manager.cc:194] Registered new tserver with Master: 7dcdd15d85ac4c96b63e180ba0210c67 (127.1.65.129:35381)
I20260812 06:36:45.588423  1638 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42034
I20260812 06:36:45.588534  1286 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.01003449s
I20260812 06:36:45.589141  1286 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 5.3411e-05s
time=2026-08-12T06:36:45.665Z level=INFO source=main.go:1678 msg="updated GOGC" old=100 new=75
time=2026-08-12T06:36:45.666Z level=INFO source=main.go:744 msg="Leaving GOMAXPROCS=4: CPU quota undefined" component=automaxprocs
time=2026-08-12T06:36:45.667Z level=INFO source=memlimit.go:198 msg="GOMEMLIMIT is updated" component=automemlimit package=github.com/KimMachineGun/automemlimit/memlimit GOMEMLIMIT=14142552883 previous=9223372036854775807
time=2026-08-12T06:36:45.667Z level=INFO source=main.go:851 msg="Starting Prometheus Server" mode=server version="(version=3.11.2, branch=HEAD, revision=f0f0fdd679dcd6df320b0558b20919f7cd44c407)"
time=2026-08-12T06:36:45.667Z level=INFO source=main.go:856 msg="operational information" build_context="(go=go1.26.2, platform=linux/amd64, user=root@d2684485c347, date=20260413-11:53:40, tags=netgo,builtinassets)" host_details="(Linux 5.4.0-1157-gcp #166~18.04.1-Ubuntu SMP Wed Jan 21 01:57:46 UTC 2026 x86_64 dist-test-slave-92m1 (none))" fd_limits="(soft=1048576, hard=1048576)" vm_limits="(soft=unlimited, hard=unlimited)"
time=2026-08-12T06:36:45.896Z level=INFO source=web.go:710 msg="Start listening for connections" component=web address=127.0.0.1:0
time=2026-08-12T06:36:45.897Z level=INFO source=main.go:1410 msg="Starting TSDB ..."
time=2026-08-12T06:36:45.901Z level=INFO source=tls_config.go:354 msg="Listening on" component=web address=127.0.0.1:38061
time=2026-08-12T06:36:45.901Z level=INFO source=tls_config.go:357 msg="TLS is disabled." component=web http2=false address=127.0.0.1:38061
time=2026-08-12T06:36:45.903Z level=INFO source=head.go:698 msg="Replaying on-disk memory mappable chunks if any" component=tsdb
time=2026-08-12T06:36:45.903Z level=INFO source=head.go:784 msg="On-disk memory mappable chunks replay completed" component=tsdb duration=2.009µs
time=2026-08-12T06:36:45.903Z level=INFO source=head.go:792 msg="Replaying WAL, this may take a while" component=tsdb
time=2026-08-12T06:36:45.904Z level=INFO source=head.go:865 msg="WAL segment loaded" component=tsdb segment=0 maxSegment=0 duration=1.22676ms
time=2026-08-12T06:36:45.904Z level=INFO source=head.go:902 msg="WAL replay completed" component=tsdb checkpoint_replay_duration=60.892µs wal_replay_duration=1.304568ms wbl_replay_duration=172ns chunk_snapshot_load_duration=0s mmap_chunk_replay_duration=2.009µs total_replay_duration=1.428583ms
time=2026-08-12T06:36:45.909Z level=INFO source=main.go:1431 msg="filesystem information" fs_type=OVERLAYFS_SUPER_MAGIC
time=2026-08-12T06:36:45.909Z level=INFO source=main.go:1434 msg="TSDB started"
time=2026-08-12T06:36:45.909Z level=INFO source=main.go:1632 msg="Loading configuration file" filename=/tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.SDLabels.1786516597789727-1286-0/prometheus.yml
time=2026-08-12T06:36:45.910Z level=INFO source=main.go:1048 msg="TSDB retention updated" duration=15d size=0B percentage=0
time=2026-08-12T06:36:45.912Z level=INFO source=main.go:1671 msg="Completed loading of configuration file" db_storage=74.764µs remote_storage=2.467µs web_handler=549ns query_engine=883ns scrape=1.762246ms scrape_sd=218.714µs notify=3.071µs notify_sd=2.268µs rules=3.543µs tracing=14.197µs filename=/tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.SDLabels.1786516597789727-1286-0/prometheus.yml totalDuration=2.546446ms
time=2026-08-12T06:36:45.912Z level=INFO source=main.go:1395 msg="Server is ready to receive web requests."
time=2026-08-12T06:36:45.912Z level=INFO source=manager.go:209 msg="Starting rule manager..." component="rule manager"
I20260812 06:36:46.589684  1865 heartbeater.cc:499] Master 127.1.65.190:37649 was elected leader, sending a full tablet report...
time=2026-08-12T06:36:52.898Z level=WARN source=main.go:1191 msg="Received an OS signal, exiting gracefully..." signal=terminated
time=2026-08-12T06:36:52.898Z level=INFO source=main.go:1216 msg="Stopping scrape discovery manager..."
time=2026-08-12T06:36:52.898Z level=INFO source=main.go:1230 msg="Stopping notify discovery manager..."
time=2026-08-12T06:36:52.898Z level=INFO source=main.go:1245 msg="Stopping rule manager manager..."
time=2026-08-12T06:36:52.898Z level=INFO source=manager.go:223 msg="Stopping rule manager..." component="rule manager"
time=2026-08-12T06:36:52.898Z level=INFO source=manager.go:239 msg="Rule manager stopped" component="rule manager"
time=2026-08-12T06:36:52.898Z level=INFO source=main.go:1269 msg="Stopping scrape manager..."
time=2026-08-12T06:36:52.898Z level=INFO source=main.go:1212 msg="Scrape discovery manager stopped"
time=2026-08-12T06:36:52.899Z level=INFO source=main.go:1226 msg="Notify discovery manager stopped"
time=2026-08-12T06:36:52.899Z level=INFO source=main.go:1241 msg="Rule manager stopped"
time=2026-08-12T06:36:52.899Z level=INFO source=main.go:1284 msg="Stopping tracing manager..."
time=2026-08-12T06:36:52.899Z level=INFO source=main.go:1261 msg="Scrape manager stopped"
time=2026-08-12T06:36:52.899Z level=INFO source=main.go:1369 msg="Stopping reloaders..."
time=2026-08-12T06:36:52.899Z level=INFO source=main.go:1458 msg="Stopping storage..."
time=2026-08-12T06:36:52.899Z level=INFO source=main.go:1361 msg="Reloaders stopped"
time=2026-08-12T06:36:52.899Z level=INFO source=main.go:1280 msg="Tracing manager stopped"
time=2026-08-12T06:36:52.904Z level=INFO source=main.go:1536 msg="Stopping web handler..."
time=2026-08-12T06:36:52.904Z level=INFO source=main.go:1454 msg="TSDB stopped"
time=2026-08-12T06:36:52.904Z level=INFO source=main.go:1559 msg="Stopping notifier manager..."
time=2026-08-12T06:36:52.904Z level=INFO source=manager.go:321 msg="Stopping notification manager..." component=notifier
time=2026-08-12T06:36:52.904Z level=INFO source=main.go:1555 msg="Notifier manager stopped"
time=2026-08-12T06:36:52.904Z level=INFO source=main.go:1532 msg="Web handler stopped"
time=2026-08-12T06:36:52.904Z level=INFO source=main.go:1570 msg="See you next time!"
I20260812 06:36:52.918540  1286 tablet_server.cc:179] TabletServer@127.1.65.129:0 shutting down...
I20260812 06:36:52.922509  1286 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260812 06:36:52.934720  1286 tablet_server.cc:196] TabletServer@127.1.65.129:0 shutdown complete.
I20260812 06:36:52.936914  1286 master.cc:562] Master@127.1.65.190:37649 shutting down...
I20260812 06:36:52.941105  1286 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 38425d33d54a44e8bdde134246ab1cb1 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:36:52.941301  1286 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 38425d33d54a44e8bdde134246ab1cb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:36:52.941383  1286 tablet_replica.cc:333] T 00000000000000000000000000000000 P 38425d33d54a44e8bdde134246ab1cb1: stopping tablet replica
I20260812 06:36:52.953724  1286 master.cc:584] Master@127.1.65.190:37649 shutdown complete.
[       OK ] PrometheusSDITest.SDLabels (7460 ms)
[ RUN      ] PrometheusSDITest.TserverMetricContinuityDuringMasterRestart
I20260812 06:36:52.958616  1286 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.1.65.190:45227
I20260812 06:36:52.958973  1286 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:36:52.960733  1907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:52.960805  1911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:52.960857  1909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:52.961163  1286 server_base.cc:1061] running on GCE node
I20260812 06:36:52.961329  1286 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 06:36:52.961365  1286 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 06:36:52.961380  1286 hybrid_clock.cc:648] HybridClock initialized: now 1786516612961381 us; error 0 us; skew 500 ppm
I20260812 06:36:52.962155  1286 webserver.cc:533] Webserver started at http://127.1.65.190:44117/ using document root <none> and password file <none>
I20260812 06:36:52.962281  1286 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:52.962319  1286 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:52.962371  1286 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:36:52.962769  1286 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786516597789727-1286-0/minicluster-data/master-0-root/instance:
uuid: "5d5cfa4139f54324ad28e9cc6f41020c"
format_stamp: "Formatted at 2026-08-12 06:36:52 on dist-test-slave-92m1"
I20260812 06:36:52.964268  1286 fs_manager.cc:696] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20260812 06:36:52.965130  1927 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:52.965395  1286 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20260812 06:36:52.965468  1286 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786516597789727-1286-0/minicluster-data/master-0-root
uuid: "5d5cfa4139f54324ad28e9cc6f41020c"
format_stamp: "Formatted at 2026-08-12 06:36:52 on dist-test-slave-92m1"
I20260812 06:36:52.965569  1286 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786516597789727-1286-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786516597789727-1286-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786516597789727-1286-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:52.973152  1286 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:52.973536  1286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:52.978355  1286 rpc_server.cc:307] RPC server started. Bound to: 127.1.65.190:45227
I20260812 06:36:52.979403  2003 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.65.190:45227 every 8 connection(s)
I20260812 06:36:52.979861  2006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:36:52.981555  2006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c: Bootstrap starting.
I20260812 06:36:52.982450  2006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c: Neither blocks nor log segments found. Creating new log.
I20260812 06:36:52.983548  2006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c: No bootstrap required, opened a new log
I20260812 06:36:52.984009  2006 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5cfa4139f54324ad28e9cc6f41020c" member_type: VOTER }
I20260812 06:36:52.984123  2006 raft_consensus.cc:385] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:36:52.984169  2006 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d5cfa4139f54324ad28e9cc6f41020c, State: Initialized, Role: FOLLOWER
I20260812 06:36:52.984344  2006 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5cfa4139f54324ad28e9cc6f41020c" member_type: VOTER }
I20260812 06:36:52.984449  2006 raft_consensus.cc:399] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:36:52.984498  2006 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:36:52.984552  2006 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:36:52.985229  2006 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5cfa4139f54324ad28e9cc6f41020c" member_type: VOTER }
I20260812 06:36:52.985380  2006 leader_election.cc:304] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d5cfa4139f54324ad28e9cc6f41020c; no voters: 
I20260812 06:36:52.985575  2006 leader_election.cc:290] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 06:36:52.985661  2010 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:36:52.985867  2010 raft_consensus.cc:697] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [term 1 LEADER]: Becoming Leader. State: Replica: 5d5cfa4139f54324ad28e9cc6f41020c, State: Running, Role: LEADER
I20260812 06:36:52.986047  2010 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5cfa4139f54324ad28e9cc6f41020c" member_type: VOTER }
I20260812 06:36:52.986078  2006 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:36:52.986519  2013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d5cfa4139f54324ad28e9cc6f41020c. Latest consensus state: current_term: 1 leader_uuid: "5d5cfa4139f54324ad28e9cc6f41020c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5cfa4139f54324ad28e9cc6f41020c" member_type: VOTER } }
I20260812 06:36:52.986591  2013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [sys.catalog]: This master's current role is: LEADER
I20260812 06:36:52.986495  2012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d5cfa4139f54324ad28e9cc6f41020c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5cfa4139f54324ad28e9cc6f41020c" member_type: VOTER } }
I20260812 06:36:52.986733  2012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [sys.catalog]: This master's current role is: LEADER
I20260812 06:36:52.986879  2015 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:36:52.987733  2015 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:36:52.988310  1286 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260812 06:36:52.989663  2015 catalog_manager.cc:1383] Generated new cluster ID: 371177574b2f4930bafb6c886221f394
I20260812 06:36:52.989733  2015 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:36:53.019767  2015 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 06:36:53.020412  2015 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:36:53.030735  2015 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c: Generated new TSK 0
I20260812 06:36:53.030912  2015 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 06:36:53.052790  1286 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:36:53.055540  2044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:53.055601  1286 server_base.cc:1061] running on GCE node
W20260812 06:36:53.055617  2046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:53.055548  2041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:53.056097  1286 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 06:36:53.056149  1286 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 06:36:53.056167  1286 hybrid_clock.cc:648] HybridClock initialized: now 1786516613056167 us; error 0 us; skew 500 ppm
I20260812 06:36:53.057097  1286 webserver.cc:533] Webserver started at http://127.1.65.129:35749/ using document root <none> and password file <none>
I20260812 06:36:53.057304  1286 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:53.057358  1286 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:53.057449  1286 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:36:53.057958  1286 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786516597789727-1286-0/minicluster-data/ts-0-root/instance:
uuid: "716d5e3863a2442ca7b8d04e1ee20245"
format_stamp: "Formatted at 2026-08-12 06:36:53 on dist-test-slave-92m1"
I20260812 06:36:53.059593  1286 fs_manager.cc:696] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20260812 06:36:53.060537  2054 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:53.060801  1286 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20260812 06:36:53.060864  1286 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786516597789727-1286-0/minicluster-data/ts-0-root
uuid: "716d5e3863a2442ca7b8d04e1ee20245"
format_stamp: "Formatted at 2026-08-12 06:36:53 on dist-test-slave-92m1"
I20260812 06:36:53.061014  1286 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786516597789727-1286-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786516597789727-1286-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786516597789727-1286-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:53.067935  1286 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:53.068298  1286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:53.068598  1286 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:36:53.069036  1286 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:36:53.069074  1286 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:53.069137  1286 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:36:53.069177  1286 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:53.073848  1286 rpc_server.cc:307] RPC server started. Bound to: 127.1.65.129:41333
I20260812 06:36:53.073882  2159 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.65.129:41333 every 8 connection(s)
I20260812 06:36:53.078683  2163 heartbeater.cc:344] Connected to a master server at 127.1.65.190:45227
I20260812 06:36:53.078785  2163 heartbeater.cc:461] Registering TS with master...
I20260812 06:36:53.079043  2163 heartbeater.cc:507] Master 127.1.65.190:45227 requested a full tablet report, sending...
I20260812 06:36:53.079763  1950 ts_manager.cc:194] Registered new tserver with Master: 716d5e3863a2442ca7b8d04e1ee20245 (127.1.65.129:41333)
I20260812 06:36:53.079985  1286 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.005646908s
I20260812 06:36:53.080480  1286 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 3.6979e-05s
I20260812 06:36:53.080682  1950 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40170
time=2026-08-12T06:36:53.156Z level=INFO source=main.go:1678 msg="updated GOGC" old=100 new=75
time=2026-08-12T06:36:53.158Z level=INFO source=main.go:744 msg="Leaving GOMAXPROCS=4: CPU quota undefined" component=automaxprocs
time=2026-08-12T06:36:53.159Z level=INFO source=memlimit.go:198 msg="GOMEMLIMIT is updated" component=automemlimit package=github.com/KimMachineGun/automemlimit/memlimit GOMEMLIMIT=14142552883 previous=9223372036854775807
time=2026-08-12T06:36:53.159Z level=INFO source=main.go:851 msg="Starting Prometheus Server" mode=server version="(version=3.11.2, branch=HEAD, revision=f0f0fdd679dcd6df320b0558b20919f7cd44c407)"
time=2026-08-12T06:36:53.159Z level=INFO source=main.go:856 msg="operational information" build_context="(go=go1.26.2, platform=linux/amd64, user=root@d2684485c347, date=20260413-11:53:40, tags=netgo,builtinassets)" host_details="(Linux 5.4.0-1157-gcp #166~18.04.1-Ubuntu SMP Wed Jan 21 01:57:46 UTC 2026 x86_64 dist-test-slave-92m1 (none))" fd_limits="(soft=1048576, hard=1048576)" vm_limits="(soft=unlimited, hard=unlimited)"
time=2026-08-12T06:36:53.391Z level=INFO source=web.go:710 msg="Start listening for connections" component=web address=127.0.0.1:0
time=2026-08-12T06:36:53.392Z level=INFO source=main.go:1410 msg="Starting TSDB ..."
time=2026-08-12T06:36:53.395Z level=INFO source=head.go:698 msg="Replaying on-disk memory mappable chunks if any" component=tsdb
time=2026-08-12T06:36:53.395Z level=INFO source=tls_config.go:354 msg="Listening on" component=web address=127.0.0.1:33923
time=2026-08-12T06:36:53.395Z level=INFO source=tls_config.go:357 msg="TLS is disabled." component=web http2=false address=127.0.0.1:33923
time=2026-08-12T06:36:53.395Z level=INFO source=head.go:784 msg="On-disk memory mappable chunks replay completed" component=tsdb duration=2.312µs
time=2026-08-12T06:36:53.395Z level=INFO source=head.go:792 msg="Replaying WAL, this may take a while" component=tsdb
time=2026-08-12T06:36:53.396Z level=INFO source=head.go:865 msg="WAL segment loaded" component=tsdb segment=0 maxSegment=0 duration=480.315µs
time=2026-08-12T06:36:53.396Z level=INFO source=head.go:902 msg="WAL replay completed" component=tsdb checkpoint_replay_duration=138.82µs wal_replay_duration=542.455µs wbl_replay_duration=244ns chunk_snapshot_load_duration=0s mmap_chunk_replay_duration=2.312µs total_replay_duration=934.946µs
time=2026-08-12T06:36:53.401Z level=INFO source=main.go:1431 msg="filesystem information" fs_type=OVERLAYFS_SUPER_MAGIC
time=2026-08-12T06:36:53.401Z level=INFO source=main.go:1434 msg="TSDB started"
time=2026-08-12T06:36:53.401Z level=INFO source=main.go:1632 msg="Loading configuration file" filename=/tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786516597789727-1286-0/prometheus.yml
time=2026-08-12T06:36:53.402Z level=INFO source=main.go:1048 msg="TSDB retention updated" duration=15d size=0B percentage=0
time=2026-08-12T06:36:53.404Z level=INFO source=main.go:1671 msg="Completed loading of configuration file" db_storage=74.08µs remote_storage=3.511µs web_handler=853ns query_engine=1.714µs scrape=1.700602ms scrape_sd=151.701µs notify=2.177µs notify_sd=1.082µs rules=2.571µs tracing=11.573µs filename=/tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786516597789727-1286-0/prometheus.yml totalDuration=2.469917ms
time=2026-08-12T06:36:53.404Z level=INFO source=main.go:1395 msg="Server is ready to receive web requests."
time=2026-08-12T06:36:53.404Z level=INFO source=manager.go:209 msg="Starting rule manager..." component="rule manager"
I20260812 06:36:54.085152  2163 heartbeater.cc:499] Master 127.1.65.190:45227 was elected leader, sending a full tablet report...
I20260812 06:37:00.377775  1286 master.cc:562] Master@127.1.65.190:45227 shutting down...
I20260812 06:37:00.381884  1286 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:37:00.382135  1286 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:37:00.382228  1286 tablet_replica.cc:333] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c: stopping tablet replica
I20260812 06:37:00.394914  1286 master.cc:584] Master@127.1.65.190:45227 shutdown complete.
I20260812 06:37:00.399494  1286 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:00.401412  2196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:00.401494  2207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:00.401587  1286 server_base.cc:1061] running on GCE node
W20260812 06:37:00.401681  2199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:00.401891  1286 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 06:37:00.401935  1286 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 06:37:00.401952  1286 hybrid_clock.cc:648] HybridClock initialized: now 1786516620401951 us; error 0 us; skew 500 ppm
I20260812 06:37:00.403024  1286 webserver.cc:533] Webserver started at http://127.1.65.190:44117/ using document root <none> and password file <none>
I20260812 06:37:00.403241  1286 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:00.403291  1286 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260812 06:37:00.404269  1286 fs_manager.cc:714] Time spent opening directory manager: real 0.001s	user 0.001s	sys 0.000s
I20260812 06:37:00.405138  2219 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:00.405400  1286 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20260812 06:37:00.405495  1286 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786516597789727-1286-0/minicluster-data/master-0-root
uuid: "5d5cfa4139f54324ad28e9cc6f41020c"
format_stamp: "Formatted at 2026-08-12 06:36:52 on dist-test-slave-92m1"
I20260812 06:37:00.405593  1286 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786516597789727-1286-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786516597789727-1286-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786516597789727-1286-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:00.419831  1286 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:00.420279  1286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:00.425318  1286 rpc_server.cc:307] RPC server started. Bound to: 127.1.65.190:45227
I20260812 06:37:00.425696  2310 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.65.190:45227 every 8 connection(s)
I20260812 06:37:00.427809  2311 sys_catalog.cc:263] Verifying existing consensus state
I20260812 06:37:00.428567  2311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c: Bootstrap starting.
I20260812 06:37:00.434175  2311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:37:00.434598  2311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c: Bootstrap complete.
I20260812 06:37:00.435101  2311 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5cfa4139f54324ad28e9cc6f41020c" member_type: VOTER }
I20260812 06:37:00.435220  2311 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d5cfa4139f54324ad28e9cc6f41020c, State: Initialized, Role: FOLLOWER
I20260812 06:37:00.435447  2311 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5cfa4139f54324ad28e9cc6f41020c" member_type: VOTER }
I20260812 06:37:00.435539  2311 raft_consensus.cc:399] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:37:00.435563  2311 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:37:00.435617  2311 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:37:00.436342  2311 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5cfa4139f54324ad28e9cc6f41020c" member_type: VOTER }
I20260812 06:37:00.436484  2311 leader_election.cc:304] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d5cfa4139f54324ad28e9cc6f41020c; no voters: 
I20260812 06:37:00.436695  2311 leader_election.cc:290] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [CANDIDATE]: Term 2 election: Requested vote from peers 
I20260812 06:37:00.436853  2317 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [term 2 FOLLOWER]: Leader election won for term 2
I20260812 06:37:00.437148  2311 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:37:00.437143  2317 raft_consensus.cc:697] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [term 2 LEADER]: Becoming Leader. State: Replica: 5d5cfa4139f54324ad28e9cc6f41020c, State: Running, Role: LEADER
I20260812 06:37:00.437448  2317 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5cfa4139f54324ad28e9cc6f41020c" member_type: VOTER }
I20260812 06:37:00.437944  2319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d5cfa4139f54324ad28e9cc6f41020c. Latest consensus state: current_term: 2 leader_uuid: "5d5cfa4139f54324ad28e9cc6f41020c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5cfa4139f54324ad28e9cc6f41020c" member_type: VOTER } }
I20260812 06:37:00.438069  2319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:00.438241  2318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5d5cfa4139f54324ad28e9cc6f41020c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d5cfa4139f54324ad28e9cc6f41020c" member_type: VOTER } }
I20260812 06:37:00.438416  2318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:00.438709  2328 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:37:00.439857  2328 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:37:00.440395  2328 catalog_manager.cc:1295] Loaded cluster ID: 371177574b2f4930bafb6c886221f394
I20260812 06:37:00.440457  2328 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:37:00.442479  2328 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:37:00.443128  2328 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c: Loaded TSK: 0
time=2026-08-12T06:37:00.443Z level=WARN source=main.go:1191 msg="Received an OS signal, exiting gracefully..." signal=terminated
time=2026-08-12T06:37:00.443Z level=INFO source=main.go:1216 msg="Stopping scrape discovery manager..."
time=2026-08-12T06:37:00.443Z level=INFO source=main.go:1230 msg="Stopping notify discovery manager..."
time=2026-08-12T06:37:00.443Z level=INFO source=main.go:1245 msg="Stopping rule manager manager..."
time=2026-08-12T06:37:00.443Z level=INFO source=manager.go:223 msg="Stopping rule manager..." component="rule manager"
time=2026-08-12T06:37:00.443Z level=INFO source=manager.go:239 msg="Rule manager stopped" component="rule manager"
time=2026-08-12T06:37:00.443Z level=INFO source=main.go:1269 msg="Stopping scrape manager..."
I20260812 06:37:00.443775  2328 catalog_manager.cc:1550] Initializing in-progress tserver states...
time=2026-08-12T06:37:00.447Z level=INFO source=main.go:1241 msg="Rule manager stopped"
time=2026-08-12T06:37:00.447Z level=INFO source=main.go:1212 msg="Scrape discovery manager stopped"
time=2026-08-12T06:37:00.447Z level=INFO source=main.go:1226 msg="Notify discovery manager stopped"
time=2026-08-12T06:37:00.447Z level=INFO source=main.go:1284 msg="Stopping tracing manager..."
time=2026-08-12T06:37:00.447Z level=INFO source=main.go:1369 msg="Stopping reloaders..."
time=2026-08-12T06:37:00.447Z level=INFO source=main.go:1458 msg="Stopping storage..."
time=2026-08-12T06:37:00.447Z level=INFO source=main.go:1261 msg="Scrape manager stopped"
time=2026-08-12T06:37:00.447Z level=INFO source=main.go:1361 msg="Reloaders stopped"
time=2026-08-12T06:37:00.447Z level=INFO source=main.go:1280 msg="Tracing manager stopped"
time=2026-08-12T06:37:00.451Z level=INFO source=main.go:1536 msg="Stopping web handler..."
time=2026-08-12T06:37:00.451Z level=INFO source=main.go:1559 msg="Stopping notifier manager..."
time=2026-08-12T06:37:00.451Z level=INFO source=manager.go:321 msg="Stopping notification manager..." component=notifier
time=2026-08-12T06:37:00.451Z level=INFO source=main.go:1454 msg="TSDB stopped"
time=2026-08-12T06:37:00.451Z level=INFO source=main.go:1555 msg="Notifier manager stopped"
time=2026-08-12T06:37:00.451Z level=INFO source=main.go:1532 msg="Web handler stopped"
time=2026-08-12T06:37:00.451Z level=INFO source=main.go:1570 msg="See you next time!"
I20260812 06:37:00.464789  1286 tablet_server.cc:179] TabletServer@127.1.65.129:0 shutting down...
I20260812 06:37:00.468715  1286 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260812 06:37:00.480576  1286 tablet_server.cc:196] TabletServer@127.1.65.129:0 shutdown complete.
I20260812 06:37:00.482496  1286 master.cc:562] Master@127.1.65.190:45227 shutting down...
I20260812 06:37:00.486294  1286 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [term 2 LEADER]: Raft consensus shutting down.
I20260812 06:37:00.486433  1286 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 5d5cfa4139f54324ad28e9cc6f41020c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 06:37:00.486541  1286 tablet_replica.cc:333] stopping tablet replica
I20260812 06:37:00.499342  1286 master.cc:584] Master@127.1.65.190:45227 shutdown complete.
[       OK ] PrometheusSDITest.TserverMetricContinuityDuringMasterRestart (7545 ms)
[----------] 3 tests from PrometheusSDITest (22705 ms total)

[----------] 2 tests from MultiMasterPrometheusSDITest
[ RUN      ] MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior
I20260812 06:37:00.504020  1286 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.1.65.190:43931,127.1.65.189:43519,127.1.65.188:33989
I20260812 06:37:00.504400  1286 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:00.505990  2339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:00.506072  2341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:00.506187  2338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:00.506480  1286 server_base.cc:1061] running on GCE node
I20260812 06:37:00.506639  1286 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 06:37:00.506686  1286 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 06:37:00.506702  1286 hybrid_clock.cc:648] HybridClock initialized: now 1786516620506703 us; error 0 us; skew 500 ppm
I20260812 06:37:00.507579  1286 webserver.cc:533] Webserver started at http://127.1.65.190:41611/ using document root <none> and password file <none>
I20260812 06:37:00.507750  1286 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:00.507814  1286 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:00.507891  1286 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:00.508286  1286 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786516597789727-1286-0/minicluster-data/master-0-root/instance:
uuid: "2aebd9929cdb439b8ef01e3836f2c58f"
format_stamp: "Formatted at 2026-08-12 06:37:00 on dist-test-slave-92m1"
I20260812 06:37:00.509800  1286 fs_manager.cc:696] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20260812 06:37:00.510763  2355 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:00.510996  1286 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20260812 06:37:00.511096  1286 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786516597789727-1286-0/minicluster-data/master-0-root
uuid: "2aebd9929cdb439b8ef01e3836f2c58f"
format_stamp: "Formatted at 2026-08-12 06:37:00 on dist-test-slave-92m1"
I20260812 06:37:00.511183  1286 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786516597789727-1286-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786516597789727-1286-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786516597789727-1286-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:00.524610  1286 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:00.524986  1286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:00.529637  1286 rpc_server.cc:307] RPC server started. Bound to: 127.1.65.190:43931
I20260812 06:37:00.530169  2437 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.65.190:43931 every 8 connection(s)
I20260812 06:37:00.530687  1286 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260812 06:37:00.530865  2438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20260812 06:37:00.532446  2445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:00.532611  2448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:00.532718  2446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:00.532871  1286 server_base.cc:1061] running on GCE node
I20260812 06:37:00.533041  1286 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260812 06:37:00.533036  2438 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43931 } has no permanent_uuid. Determining permanent_uuid...
W20260812 06:37:00.533154  1286 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 06:37:00.533218  1286 hybrid_clock.cc:648] HybridClock initialized: now 1786516620533217 us; error 0 us; skew 500 ppm
I20260812 06:37:00.534106  1286 webserver.cc:533] Webserver started at http://127.1.65.189:42643/ using document root <none> and password file <none>
I20260812 06:37:00.534286  1286 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:00.534354  1286 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:00.534430  1286 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:00.534786  1286 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786516597789727-1286-0/minicluster-data/master-1-root/instance:
uuid: "87e2eee6874b4a9a9f18c8d3a287d596"
format_stamp: "Formatted at 2026-08-12 06:37:00 on dist-test-slave-92m1"
I20260812 06:37:00.536396  1286 fs_manager.cc:696] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20260812 06:37:00.537515  2457 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:00.537750  1286 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20260812 06:37:00.537843  1286 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786516597789727-1286-0/minicluster-data/master-1-root
uuid: "87e2eee6874b4a9a9f18c8d3a287d596"
format_stamp: "Formatted at 2026-08-12 06:37:00 on dist-test-slave-92m1"
I20260812 06:37:00.537930  1286 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786516597789727-1286-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786516597789727-1286-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786516597789727-1286-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:00.542249  2438 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.65.189" port: 43519 } has no permanent_uuid. Determining permanent_uuid...
W20260812 06:37:00.542959  2362 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.65.189:43519: connect: Connection refused (error 111)
W20260812 06:37:00.543833  2438 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.1.65.189:43519: Network error: Client connection negotiation failed: client connection to 127.1.65.189:43519: connect: Connection refused (error 111)
I20260812 06:37:00.546391  1286 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:00.546727  1286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:00.550894  1286 rpc_server.cc:307] RPC server started. Bound to: 127.1.65.189:43519
I20260812 06:37:00.550947  2539 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.65.189:43519 every 8 connection(s)
I20260812 06:37:00.551612  1286 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260812 06:37:00.551914  2540 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20260812 06:37:00.553406  2545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:00.553444  2546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:00.553578  2553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:00.553642  1286 server_base.cc:1061] running on GCE node
I20260812 06:37:00.553781  2540 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43931 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:00.553828  1286 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 06:37:00.553894  1286 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 06:37:00.553920  1286 hybrid_clock.cc:648] HybridClock initialized: now 1786516620553919 us; error 0 us; skew 500 ppm
I20260812 06:37:00.554811  1286 webserver.cc:533] Webserver started at http://127.1.65.188:37669/ using document root <none> and password file <none>
I20260812 06:37:00.555020  1286 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:00.555130  1286 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:00.555260  1286 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:00.555704  1286 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786516597789727-1286-0/minicluster-data/master-2-root/instance:
uuid: "c70fc4027e7d4cb29f239aa884f53498"
format_stamp: "Formatted at 2026-08-12 06:37:00 on dist-test-slave-92m1"
I20260812 06:37:00.557557  1286 fs_manager.cc:696] Time spent creating directory manager: real 0.002s	user 0.000s	sys 0.002s
I20260812 06:37:00.558406  2540 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.65.189" port: 43519 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:00.558898  2572 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:00.559154  1286 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.001s
I20260812 06:37:00.559254  1286 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786516597789727-1286-0/minicluster-data/master-2-root
uuid: "c70fc4027e7d4cb29f239aa884f53498"
format_stamp: "Formatted at 2026-08-12 06:37:00 on dist-test-slave-92m1"
I20260812 06:37:00.559334  1286 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786516597789727-1286-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786516597789727-1286-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786516597789727-1286-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:00.562829  2540 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.65.188" port: 33989 } has no permanent_uuid. Determining permanent_uuid...
W20260812 06:37:00.564163  2540 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.1.65.188:33989: Network error: Client connection negotiation failed: client connection to 127.1.65.188:33989: connect: Connection refused (error 111)
I20260812 06:37:00.589535  1286 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:00.589983  1286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:00.595139  1286 rpc_server.cc:307] RPC server started. Bound to: 127.1.65.188:33989
I20260812 06:37:00.595253  2661 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.65.188:33989 every 8 connection(s)
I20260812 06:37:00.595671  1286 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260812 06:37:00.596181  2664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:00.597424  2664 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43931 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:00.602041  2664 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.65.189" port: 43519 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:00.603031  2438 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.65.189" port: 43519 } attempt: 1
I20260812 06:37:00.606112  2664 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.65.188" port: 33989 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:00.607173  2438 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.65.188" port: 33989 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:00.611753  2664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70fc4027e7d4cb29f239aa884f53498: Bootstrap starting.
I20260812 06:37:00.611944  2438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f: Bootstrap starting.
I20260812 06:37:00.612870  2438 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:00.612869  2664 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c70fc4027e7d4cb29f239aa884f53498: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:00.613988  2664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70fc4027e7d4cb29f239aa884f53498: No bootstrap required, opened a new log
I20260812 06:37:00.613991  2438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f: No bootstrap required, opened a new log
I20260812 06:37:00.614465  2664 raft_consensus.cc:359] T 00000000000000000000000000000000 P c70fc4027e7d4cb29f239aa884f53498 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43931 } } peers { permanent_uuid: "87e2eee6874b4a9a9f18c8d3a287d596" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 43519 } } peers { permanent_uuid: "c70fc4027e7d4cb29f239aa884f53498" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 33989 } }
I20260812 06:37:00.614555  2664 raft_consensus.cc:385] T 00000000000000000000000000000000 P c70fc4027e7d4cb29f239aa884f53498 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:00.614478  2438 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43931 } } peers { permanent_uuid: "87e2eee6874b4a9a9f18c8d3a287d596" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 43519 } } peers { permanent_uuid: "c70fc4027e7d4cb29f239aa884f53498" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 33989 } }
I20260812 06:37:00.614584  2664 raft_consensus.cc:740] T 00000000000000000000000000000000 P c70fc4027e7d4cb29f239aa884f53498 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c70fc4027e7d4cb29f239aa884f53498, State: Initialized, Role: FOLLOWER
I20260812 06:37:00.614609  2438 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:00.614670  2438 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2aebd9929cdb439b8ef01e3836f2c58f, State: Initialized, Role: FOLLOWER
I20260812 06:37:00.614784  2664 consensus_queue.cc:260] T 00000000000000000000000000000000 P c70fc4027e7d4cb29f239aa884f53498 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43931 } } peers { permanent_uuid: "87e2eee6874b4a9a9f18c8d3a287d596" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 43519 } } peers { permanent_uuid: "c70fc4027e7d4cb29f239aa884f53498" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 33989 } }
I20260812 06:37:00.614856  2438 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43931 } } peers { permanent_uuid: "87e2eee6874b4a9a9f18c8d3a287d596" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 43519 } } peers { permanent_uuid: "c70fc4027e7d4cb29f239aa884f53498" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 33989 } }
I20260812 06:37:00.615157  2678 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70fc4027e7d4cb29f239aa884f53498 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43931 } } peers { permanent_uuid: "87e2eee6874b4a9a9f18c8d3a287d596" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 43519 } } peers { permanent_uuid: "c70fc4027e7d4cb29f239aa884f53498" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 33989 } } }
I20260812 06:37:00.615192  2682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43931 } } peers { permanent_uuid: "87e2eee6874b4a9a9f18c8d3a287d596" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 43519 } } peers { permanent_uuid: "c70fc4027e7d4cb29f239aa884f53498" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 33989 } } }
I20260812 06:37:00.615334  2540 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.65.188" port: 33989 } attempt: 1
I20260812 06:37:00.615411  2678 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70fc4027e7d4cb29f239aa884f53498 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:00.615420  2438 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:37:00.615572  2682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:00.617439  1286 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
I20260812 06:37:00.615422  2664 sys_catalog.cc:565] T 00000000000000000000000000000000 P c70fc4027e7d4cb29f239aa884f53498 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 06:37:00.618063  2698 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:37:00.618139  2698 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260812 06:37:00.620153  2721 catalog_manager.cc:1594] T 00000000000000000000000000000000 P c70fc4027e7d4cb29f239aa884f53498: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:37:00.620222  2721 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:37:00.620436  2540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87e2eee6874b4a9a9f18c8d3a287d596: Bootstrap starting.
I20260812 06:37:00.621212  2540 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87e2eee6874b4a9a9f18c8d3a287d596: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:00.622143  2540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87e2eee6874b4a9a9f18c8d3a287d596: No bootstrap required, opened a new log
I20260812 06:37:00.622540  2540 raft_consensus.cc:359] T 00000000000000000000000000000000 P 87e2eee6874b4a9a9f18c8d3a287d596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43931 } } peers { permanent_uuid: "87e2eee6874b4a9a9f18c8d3a287d596" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 43519 } } peers { permanent_uuid: "c70fc4027e7d4cb29f239aa884f53498" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 33989 } }
I20260812 06:37:00.622618  2540 raft_consensus.cc:385] T 00000000000000000000000000000000 P 87e2eee6874b4a9a9f18c8d3a287d596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:00.622640  2540 raft_consensus.cc:740] T 00000000000000000000000000000000 P 87e2eee6874b4a9a9f18c8d3a287d596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87e2eee6874b4a9a9f18c8d3a287d596, State: Initialized, Role: FOLLOWER
I20260812 06:37:00.622766  2540 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87e2eee6874b4a9a9f18c8d3a287d596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43931 } } peers { permanent_uuid: "87e2eee6874b4a9a9f18c8d3a287d596" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 43519 } } peers { permanent_uuid: "c70fc4027e7d4cb29f239aa884f53498" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 33989 } }
I20260812 06:37:00.623057  2723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87e2eee6874b4a9a9f18c8d3a287d596 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43931 } } peers { permanent_uuid: "87e2eee6874b4a9a9f18c8d3a287d596" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 43519 } } peers { permanent_uuid: "c70fc4027e7d4cb29f239aa884f53498" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 33989 } } }
I20260812 06:37:00.623150  2540 sys_catalog.cc:565] T 00000000000000000000000000000000 P 87e2eee6874b4a9a9f18c8d3a287d596 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:37:00.623168  2723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87e2eee6874b4a9a9f18c8d3a287d596 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:00.624755  1286 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20260812 06:37:00.625216  1286 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:00.625344  2744 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 87e2eee6874b4a9a9f18c8d3a287d596: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:37:00.625399  2744 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260812 06:37:00.627292  2750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:00.627352  2747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:00.627368  2745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:00.627470  1286 server_base.cc:1061] running on GCE node
I20260812 06:37:00.627698  1286 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 06:37:00.627739  1286 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 06:37:00.627755  1286 hybrid_clock.cc:648] HybridClock initialized: now 1786516620627756 us; error 0 us; skew 500 ppm
I20260812 06:37:00.628543  1286 webserver.cc:533] Webserver started at http://127.1.65.129:33127/ using document root <none> and password file <none>
I20260812 06:37:00.628679  1286 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:00.628717  1286 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:00.628793  1286 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:00.629096  1286 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786516597789727-1286-0/minicluster-data/ts-0-root/instance:
uuid: "9711153af9044efebdb13419e378ad30"
format_stamp: "Formatted at 2026-08-12 06:37:00 on dist-test-slave-92m1"
I20260812 06:37:00.630456  1286 fs_manager.cc:696] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20260812 06:37:00.631313  2757 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:00.631562  1286 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20260812 06:37:00.631651  1286 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786516597789727-1286-0/minicluster-data/ts-0-root
uuid: "9711153af9044efebdb13419e378ad30"
format_stamp: "Formatted at 2026-08-12 06:37:00 on dist-test-slave-92m1"
I20260812 06:37:00.631718  1286 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786516597789727-1286-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786516597789727-1286-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786516597789727-1286-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:00.653942  1286 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:00.654369  1286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:00.654683  1286 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:00.655319  1286 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:00.655380  1286 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:00.655437  1286 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:00.655486  1286 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:00.661835  1286 rpc_server.cc:307] RPC server started. Bound to: 127.1.65.129:35381
I20260812 06:37:00.661892  2870 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.65.129:35381 every 8 connection(s)
I20260812 06:37:00.667302  2871 heartbeater.cc:344] Connected to a master server at 127.1.65.188:33989
I20260812 06:37:00.667361  2874 heartbeater.cc:344] Connected to a master server at 127.1.65.189:43519
I20260812 06:37:00.667408  2874 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:00.667408  2871 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:00.667584  2872 heartbeater.cc:344] Connected to a master server at 127.1.65.190:43931
I20260812 06:37:00.667649  2874 heartbeater.cc:507] Master 127.1.65.189:43519 requested a full tablet report, sending...
I20260812 06:37:00.667649  2871 heartbeater.cc:507] Master 127.1.65.188:33989 requested a full tablet report, sending...
I20260812 06:37:00.667671  2872 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:00.667955  2872 heartbeater.cc:507] Master 127.1.65.190:43931 requested a full tablet report, sending...
I20260812 06:37:00.668429  2487 ts_manager.cc:194] Registered new tserver with Master: 9711153af9044efebdb13419e378ad30 (127.1.65.129:35381)
I20260812 06:37:00.668500  2382 ts_manager.cc:194] Registered new tserver with Master: 9711153af9044efebdb13419e378ad30 (127.1.65.129:35381)
I20260812 06:37:00.668802  2605 ts_manager.cc:194] Registered new tserver with Master: 9711153af9044efebdb13419e378ad30 (127.1.65.129:35381)
I20260812 06:37:00.669291  1286 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.006923506s
I20260812 06:37:00.669847  1286 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 3.9397e-05s
I20260812 06:37:00.727409  2682 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:37:00.727648  2682 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43931 } } peers { permanent_uuid: "87e2eee6874b4a9a9f18c8d3a287d596" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 43519 } } peers { permanent_uuid: "c70fc4027e7d4cb29f239aa884f53498" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 33989 } }
I20260812 06:37:00.728749  2682 leader_election.cc:290] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87e2eee6874b4a9a9f18c8d3a287d596 (127.1.65.189:43519), c70fc4027e7d4cb29f239aa884f53498 (127.1.65.188:33989)
I20260812 06:37:00.729499  2503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87e2eee6874b4a9a9f18c8d3a287d596" is_pre_election: true
I20260812 06:37:00.729496  2622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c70fc4027e7d4cb29f239aa884f53498" is_pre_election: true
I20260812 06:37:00.729862  2622 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c70fc4027e7d4cb29f239aa884f53498 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2aebd9929cdb439b8ef01e3836f2c58f in term 0.
I20260812 06:37:00.730261  2362 leader_election.cc:304] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2aebd9929cdb439b8ef01e3836f2c58f, c70fc4027e7d4cb29f239aa884f53498; no voters: 
I20260812 06:37:00.730404  2682 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:37:00.730450  2682 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:37:00.730477  2682 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:00.731101  2503 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 87e2eee6874b4a9a9f18c8d3a287d596 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2aebd9929cdb439b8ef01e3836f2c58f in term 0.
I20260812 06:37:00.731658  2682 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43931 } } peers { permanent_uuid: "87e2eee6874b4a9a9f18c8d3a287d596" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 43519 } } peers { permanent_uuid: "c70fc4027e7d4cb29f239aa884f53498" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 33989 } }
I20260812 06:37:00.732035  2682 leader_election.cc:290] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [CANDIDATE]: Term 1 election: Requested vote from peers 87e2eee6874b4a9a9f18c8d3a287d596 (127.1.65.189:43519), c70fc4027e7d4cb29f239aa884f53498 (127.1.65.188:33989)
I20260812 06:37:00.732437  2503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87e2eee6874b4a9a9f18c8d3a287d596"
I20260812 06:37:00.732544  2622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c70fc4027e7d4cb29f239aa884f53498"
I20260812 06:37:00.732570  2503 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 87e2eee6874b4a9a9f18c8d3a287d596 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:00.732643  2622 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c70fc4027e7d4cb29f239aa884f53498 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:00.733634  2622 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c70fc4027e7d4cb29f239aa884f53498 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2aebd9929cdb439b8ef01e3836f2c58f in term 1.
I20260812 06:37:00.733634  2503 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 87e2eee6874b4a9a9f18c8d3a287d596 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2aebd9929cdb439b8ef01e3836f2c58f in term 1.
I20260812 06:37:00.734217  2362 leader_election.cc:304] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2aebd9929cdb439b8ef01e3836f2c58f, 87e2eee6874b4a9a9f18c8d3a287d596; no voters: 
I20260812 06:37:00.734508  2682 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:37:00.734771  2682 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [term 1 LEADER]: Becoming Leader. State: Replica: 2aebd9929cdb439b8ef01e3836f2c58f, State: Running, Role: LEADER
I20260812 06:37:00.734956  2682 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43931 } } peers { permanent_uuid: "87e2eee6874b4a9a9f18c8d3a287d596" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 43519 } } peers { permanent_uuid: "c70fc4027e7d4cb29f239aa884f53498" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 33989 } }
I20260812 06:37:00.736514  2897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2aebd9929cdb439b8ef01e3836f2c58f. Latest consensus state: current_term: 1 leader_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43931 } } peers { permanent_uuid: "87e2eee6874b4a9a9f18c8d3a287d596" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 43519 } } peers { permanent_uuid: "c70fc4027e7d4cb29f239aa884f53498" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 33989 } } }
I20260812 06:37:00.736644  2897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [sys.catalog]: This master's current role is: LEADER
time=2026-08-12T06:37:00.736Z level=INFO source=main.go:1678 msg="updated GOGC" old=100 new=75
I20260812 06:37:00.738557  2900 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
time=2026-08-12T06:37:00.740Z level=INFO source=main.go:744 msg="Leaving GOMAXPROCS=4: CPU quota undefined" component=automaxprocs
I20260812 06:37:00.740172  2900 catalog_manager.cc:1520] Initializing Kudu cluster ID...
time=2026-08-12T06:37:00.740Z level=INFO source=memlimit.go:198 msg="GOMEMLIMIT is updated" component=automemlimit package=github.com/KimMachineGun/automemlimit/memlimit GOMEMLIMIT=14142552883 previous=9223372036854775807
time=2026-08-12T06:37:00.740Z level=INFO source=main.go:851 msg="Starting Prometheus Server" mode=server version="(version=3.11.2, branch=HEAD, revision=f0f0fdd679dcd6df320b0558b20919f7cd44c407)"
time=2026-08-12T06:37:00.741Z level=INFO source=main.go:856 msg="operational information" build_context="(go=go1.26.2, platform=linux/amd64, user=root@d2684485c347, date=20260413-11:53:40, tags=netgo,builtinassets)" host_details="(Linux 5.4.0-1157-gcp #166~18.04.1-Ubuntu SMP Wed Jan 21 01:57:46 UTC 2026 x86_64 dist-test-slave-92m1 (none))" fd_limits="(soft=1048576, hard=1048576)" vm_limits="(soft=unlimited, hard=unlimited)"
I20260812 06:37:00.743806  2503 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 87e2eee6874b4a9a9f18c8d3a287d596 [term 1 FOLLOWER]: Refusing update from remote peer 2aebd9929cdb439b8ef01e3836f2c58f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:37:00.744264  2622 raft_consensus.cc:1275] T 00000000000000000000000000000000 P c70fc4027e7d4cb29f239aa884f53498 [term 1 FOLLOWER]: Refusing update from remote peer 2aebd9929cdb439b8ef01e3836f2c58f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:37:00.744581  2897 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [LEADER]: Connected to new peer: Peer: permanent_uuid: "87e2eee6874b4a9a9f18c8d3a287d596" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:37:00.744714  2682 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c70fc4027e7d4cb29f239aa884f53498" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 33989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:37:00.747237  2678 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70fc4027e7d4cb29f239aa884f53498 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2aebd9929cdb439b8ef01e3836f2c58f. Latest consensus state: current_term: 1 leader_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43931 } } peers { permanent_uuid: "87e2eee6874b4a9a9f18c8d3a287d596" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 43519 } } peers { permanent_uuid: "c70fc4027e7d4cb29f239aa884f53498" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 33989 } } }
I20260812 06:37:00.747362  2723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87e2eee6874b4a9a9f18c8d3a287d596 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2aebd9929cdb439b8ef01e3836f2c58f. Latest consensus state: current_term: 1 leader_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43931 } } peers { permanent_uuid: "87e2eee6874b4a9a9f18c8d3a287d596" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 43519 } } peers { permanent_uuid: "c70fc4027e7d4cb29f239aa884f53498" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 33989 } } }
I20260812 06:37:00.747545  2723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87e2eee6874b4a9a9f18c8d3a287d596 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:00.747370  2678 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70fc4027e7d4cb29f239aa884f53498 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:00.749666  2901 mvcc.cc:204] Tried to move back new op lower bound from 7317572078558474240 to 7317572078534721536. Current Snapshot: MvccSnapshot[applied={T|T < 7317572078558474240}]
I20260812 06:37:00.750217  2682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43931 } } peers { permanent_uuid: "87e2eee6874b4a9a9f18c8d3a287d596" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 43519 } } peers { permanent_uuid: "c70fc4027e7d4cb29f239aa884f53498" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 33989 } } }
I20260812 06:37:00.750330  2682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:00.750839  2723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87e2eee6874b4a9a9f18c8d3a287d596 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43931 } } peers { permanent_uuid: "87e2eee6874b4a9a9f18c8d3a287d596" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 43519 } } peers { permanent_uuid: "c70fc4027e7d4cb29f239aa884f53498" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 33989 } } }
I20260812 06:37:00.750953  2723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87e2eee6874b4a9a9f18c8d3a287d596 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:00.754108  2682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43931 } } peers { permanent_uuid: "87e2eee6874b4a9a9f18c8d3a287d596" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 43519 } } peers { permanent_uuid: "c70fc4027e7d4cb29f239aa884f53498" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 33989 } } }
I20260812 06:37:00.754213  2682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:00.755139  2678 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70fc4027e7d4cb29f239aa884f53498 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aebd9929cdb439b8ef01e3836f2c58f" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43931 } } peers { permanent_uuid: "87e2eee6874b4a9a9f18c8d3a287d596" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 43519 } } peers { permanent_uuid: "c70fc4027e7d4cb29f239aa884f53498" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 33989 } } }
I20260812 06:37:00.755282  2678 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70fc4027e7d4cb29f239aa884f53498 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:00.759871  2900 catalog_manager.cc:1383] Generated new cluster ID: 3303a33d61224e8998ffdf7f177a666b
I20260812 06:37:00.759964  2900 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:37:00.795861  2900 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 06:37:00.796756  2900 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:37:00.813987  2900 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f: Generated new TSK 0
I20260812 06:37:00.814168  2900 catalog_manager.cc:1550] Initializing in-progress tserver states...
time=2026-08-12T06:37:01.000Z level=INFO source=web.go:710 msg="Start listening for connections" component=web address=127.0.0.1:0
time=2026-08-12T06:37:01.001Z level=INFO source=main.go:1410 msg="Starting TSDB ..."
time=2026-08-12T06:37:01.005Z level=INFO source=tls_config.go:354 msg="Listening on" component=web address=127.0.0.1:33629
time=2026-08-12T06:37:01.005Z level=INFO source=tls_config.go:357 msg="TLS is disabled." component=web http2=false address=127.0.0.1:33629
time=2026-08-12T06:37:01.005Z level=INFO source=head.go:698 msg="Replaying on-disk memory mappable chunks if any" component=tsdb
time=2026-08-12T06:37:01.005Z level=INFO source=head.go:784 msg="On-disk memory mappable chunks replay completed" component=tsdb duration=2.207µs
time=2026-08-12T06:37:01.005Z level=INFO source=head.go:792 msg="Replaying WAL, this may take a while" component=tsdb
time=2026-08-12T06:37:01.006Z level=INFO source=head.go:865 msg="WAL segment loaded" component=tsdb segment=0 maxSegment=0 duration=1.420931ms
time=2026-08-12T06:37:01.006Z level=INFO source=head.go:902 msg="WAL replay completed" component=tsdb checkpoint_replay_duration=62.119µs wal_replay_duration=1.465621ms wbl_replay_duration=379ns chunk_snapshot_load_duration=0s mmap_chunk_replay_duration=2.207µs total_replay_duration=1.586938ms
time=2026-08-12T06:37:01.008Z level=INFO source=main.go:1431 msg="filesystem information" fs_type=OVERLAYFS_SUPER_MAGIC
time=2026-08-12T06:37:01.008Z level=INFO source=main.go:1434 msg="TSDB started"
time=2026-08-12T06:37:01.008Z level=INFO source=main.go:1632 msg="Loading configuration file" filename=/tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786516597789727-1286-0/prometheus.yml
time=2026-08-12T06:37:01.009Z level=INFO source=main.go:1048 msg="TSDB retention updated" duration=15d size=0B percentage=0
time=2026-08-12T06:37:01.011Z level=INFO source=main.go:1671 msg="Completed loading of configuration file" db_storage=77.165µs remote_storage=2.395µs web_handler=744ns query_engine=985ns scrape=1.714149ms scrape_sd=163.548µs notify=2.342µs notify_sd=1.093µs rules=1.973µs tracing=5.289µs filename=/tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786516597789727-1286-0/prometheus.yml totalDuration=2.310388ms
time=2026-08-12T06:37:01.011Z level=INFO source=main.go:1395 msg="Server is ready to receive web requests."
time=2026-08-12T06:37:01.011Z level=INFO source=manager.go:209 msg="Starting rule manager..." component="rule manager"
I20260812 06:37:01.621659  2721 catalog_manager.cc:1295] Loaded cluster ID: 3303a33d61224e8998ffdf7f177a666b
I20260812 06:37:01.621805  2721 catalog_manager.cc:1588] T 00000000000000000000000000000000 P c70fc4027e7d4cb29f239aa884f53498: loading cluster ID for follower catalog manager: success
I20260812 06:37:01.624368  2721 catalog_manager.cc:1610] T 00000000000000000000000000000000 P c70fc4027e7d4cb29f239aa884f53498: acquiring CA information for follower catalog manager: success
I20260812 06:37:01.625552  2721 catalog_manager.cc:1638] T 00000000000000000000000000000000 P c70fc4027e7d4cb29f239aa884f53498: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:37:01.626049  2744 catalog_manager.cc:1295] Loaded cluster ID: 3303a33d61224e8998ffdf7f177a666b
I20260812 06:37:01.626139  2744 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 87e2eee6874b4a9a9f18c8d3a287d596: loading cluster ID for follower catalog manager: success
I20260812 06:37:01.627214  2744 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 87e2eee6874b4a9a9f18c8d3a287d596: acquiring CA information for follower catalog manager: success
I20260812 06:37:01.627751  2744 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 87e2eee6874b4a9a9f18c8d3a287d596: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:37:01.671528  2382 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54486
I20260812 06:37:02.672649  2872 heartbeater.cc:499] Master 127.1.65.190:43931 was elected leader, sending a full tablet report...
time=2026-08-12T06:37:08.047Z level=WARN source=main.go:1191 msg="Received an OS signal, exiting gracefully..." signal=terminated
time=2026-08-12T06:37:08.047Z level=INFO source=main.go:1216 msg="Stopping scrape discovery manager..."
time=2026-08-12T06:37:08.047Z level=INFO source=main.go:1230 msg="Stopping notify discovery manager..."
time=2026-08-12T06:37:08.047Z level=INFO source=main.go:1245 msg="Stopping rule manager manager..."
time=2026-08-12T06:37:08.047Z level=INFO source=manager.go:223 msg="Stopping rule manager..." component="rule manager"
time=2026-08-12T06:37:08.047Z level=INFO source=manager.go:239 msg="Rule manager stopped" component="rule manager"
time=2026-08-12T06:37:08.047Z level=INFO source=main.go:1269 msg="Stopping scrape manager..."
time=2026-08-12T06:37:08.055Z level=INFO source=main.go:1241 msg="Rule manager stopped"
time=2026-08-12T06:37:08.047Z level=INFO source=main.go:1212 msg="Scrape discovery manager stopped"
time=2026-08-12T06:37:08.055Z level=INFO source=main.go:1284 msg="Stopping tracing manager..."
time=2026-08-12T06:37:08.055Z level=INFO source=main.go:1369 msg="Stopping reloaders..."
time=2026-08-12T06:37:08.055Z level=INFO source=main.go:1458 msg="Stopping storage..."
time=2026-08-12T06:37:08.055Z level=INFO source=main.go:1226 msg="Notify discovery manager stopped"
time=2026-08-12T06:37:08.055Z level=INFO source=main.go:1280 msg="Tracing manager stopped"
time=2026-08-12T06:37:08.055Z level=INFO source=main.go:1361 msg="Reloaders stopped"
time=2026-08-12T06:37:08.056Z level=INFO source=main.go:1261 msg="Scrape manager stopped"
time=2026-08-12T06:37:08.060Z level=INFO source=main.go:1536 msg="Stopping web handler..."
time=2026-08-12T06:37:08.060Z level=INFO source=main.go:1559 msg="Stopping notifier manager..."
time=2026-08-12T06:37:08.060Z level=INFO source=manager.go:321 msg="Stopping notification manager..." component=notifier
time=2026-08-12T06:37:08.060Z level=INFO source=main.go:1555 msg="Notifier manager stopped"
time=2026-08-12T06:37:08.060Z level=INFO source=main.go:1454 msg="TSDB stopped"
time=2026-08-12T06:37:08.060Z level=INFO source=main.go:1532 msg="Web handler stopped"
time=2026-08-12T06:37:08.060Z level=INFO source=main.go:1570 msg="See you next time!"
I20260812 06:37:08.078074  1286 tablet_server.cc:179] TabletServer@127.1.65.129:0 shutting down...
I20260812 06:37:08.082712  1286 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260812 06:37:08.095788  1286 tablet_server.cc:196] TabletServer@127.1.65.129:0 shutdown complete.
I20260812 06:37:08.097868  1286 master.cc:562] Master@127.1.65.190:43931 shutting down...
I20260812 06:37:08.101549  1286 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:37:08.101940  1286 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:37:08.102044  1286 tablet_replica.cc:333] T 00000000000000000000000000000000 P 2aebd9929cdb439b8ef01e3836f2c58f: stopping tablet replica
I20260812 06:37:08.114945  1286 master.cc:584] Master@127.1.65.190:43931 shutdown complete.
I20260812 06:37:08.117640  1286 master.cc:562] Master@127.1.65.189:43519 shutting down...
I20260812 06:37:08.122531  1286 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 87e2eee6874b4a9a9f18c8d3a287d596 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:37:08.122692  1286 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 87e2eee6874b4a9a9f18c8d3a287d596 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:37:08.122761  1286 tablet_replica.cc:333] T 00000000000000000000000000000000 P 87e2eee6874b4a9a9f18c8d3a287d596: stopping tablet replica
I20260812 06:37:08.135596  1286 master.cc:584] Master@127.1.65.189:43519 shutdown complete.
I20260812 06:37:08.138084  1286 master.cc:562] Master@127.1.65.188:33989 shutting down...
I20260812 06:37:08.142355  1286 raft_consensus.cc:2243] T 00000000000000000000000000000000 P c70fc4027e7d4cb29f239aa884f53498 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:37:08.142491  1286 raft_consensus.cc:2272] T 00000000000000000000000000000000 P c70fc4027e7d4cb29f239aa884f53498 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:37:08.142534  1286 tablet_replica.cc:333] T 00000000000000000000000000000000 P c70fc4027e7d4cb29f239aa884f53498: stopping tablet replica
I20260812 06:37:08.154829  1286 master.cc:584] Master@127.1.65.188:33989 shutdown complete.
[       OK ] MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior (7656 ms)
[ RUN      ] MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower
I20260812 06:37:08.160953  1286 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.1.65.190:43845,127.1.65.189:45799,127.1.65.188:46273
I20260812 06:37:08.161350  1286 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:08.163247  2933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:08.163313  2934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:08.163367  2937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:08.163470  1286 server_base.cc:1061] running on GCE node
I20260812 06:37:08.163652  1286 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 06:37:08.163692  1286 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 06:37:08.163708  1286 hybrid_clock.cc:648] HybridClock initialized: now 1786516628163708 us; error 0 us; skew 500 ppm
I20260812 06:37:08.164501  1286 webserver.cc:533] Webserver started at http://127.1.65.190:40221/ using document root <none> and password file <none>
I20260812 06:37:08.164644  1286 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:08.164687  1286 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:08.164740  1286 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:08.165055  1286 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786516597789727-1286-0/minicluster-data/master-0-root/instance:
uuid: "f281bbbdadfb4e87bb58276193f5be13"
format_stamp: "Formatted at 2026-08-12 06:37:08 on dist-test-slave-92m1"
I20260812 06:37:08.166729  1286 fs_manager.cc:696] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20260812 06:37:08.167783  2946 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:08.168032  1286 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.000s
I20260812 06:37:08.168123  1286 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786516597789727-1286-0/minicluster-data/master-0-root
uuid: "f281bbbdadfb4e87bb58276193f5be13"
format_stamp: "Formatted at 2026-08-12 06:37:08 on dist-test-slave-92m1"
I20260812 06:37:08.168211  1286 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786516597789727-1286-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786516597789727-1286-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786516597789727-1286-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:08.174834  1286 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:08.175172  1286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:08.179730  1286 rpc_server.cc:307] RPC server started. Bound to: 127.1.65.190:43845
I20260812 06:37:08.181016  1286 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260812 06:37:08.183418  3031 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.65.190:43845 every 8 connection(s)
I20260812 06:37:08.183941  3032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20260812 06:37:08.184435  3035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:08.184453  3043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:08.184512  3037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:08.184808  1286 server_base.cc:1061] running on GCE node
I20260812 06:37:08.184930  1286 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 06:37:08.184963  1286 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 06:37:08.184979  1286 hybrid_clock.cc:648] HybridClock initialized: now 1786516628184979 us; error 0 us; skew 500 ppm
I20260812 06:37:08.185400  3032 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43845 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:08.185815  1286 webserver.cc:533] Webserver started at http://127.1.65.189:33863/ using document root <none> and password file <none>
I20260812 06:37:08.185974  1286 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:08.186033  1286 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:08.186105  1286 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:08.186425  1286 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786516597789727-1286-0/minicluster-data/master-1-root/instance:
uuid: "c6d40a4a3831497cbff775c15b18a6f0"
format_stamp: "Formatted at 2026-08-12 06:37:08 on dist-test-slave-92m1"
I20260812 06:37:08.188035  1286 fs_manager.cc:696] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20260812 06:37:08.189134  3054 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:08.189429  1286 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20260812 06:37:08.189522  1286 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786516597789727-1286-0/minicluster-data/master-1-root
uuid: "c6d40a4a3831497cbff775c15b18a6f0"
format_stamp: "Formatted at 2026-08-12 06:37:08 on dist-test-slave-92m1"
I20260812 06:37:08.189592  1286 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786516597789727-1286-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786516597789727-1286-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786516597789727-1286-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:08.191991  3032 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.65.189" port: 45799 } has no permanent_uuid. Determining permanent_uuid...
W20260812 06:37:08.192683  2954 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.65.189:45799: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260812 06:37:08.193416  3032 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.1.65.189:45799: Network error: Client connection negotiation failed: client connection to 127.1.65.189:45799: connect: Connection refused (error 111)
I20260812 06:37:08.217541  1286 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:08.217962  1286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:08.222776  1286 rpc_server.cc:307] RPC server started. Bound to: 127.1.65.189:45799
I20260812 06:37:08.222808  3163 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.65.189:45799 every 8 connection(s)
I20260812 06:37:08.223534  1286 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260812 06:37:08.223694  3164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20260812 06:37:08.225208  3166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:08.225323  3169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:08.225306  3164 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43845 } has no permanent_uuid. Determining permanent_uuid...
W20260812 06:37:08.225306  3167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:08.225771  1286 server_base.cc:1061] running on GCE node
I20260812 06:37:08.226019  1286 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 06:37:08.226073  1286 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 06:37:08.226100  1286 hybrid_clock.cc:648] HybridClock initialized: now 1786516628226100 us; error 0 us; skew 500 ppm
I20260812 06:37:08.226974  1286 webserver.cc:533] Webserver started at http://127.1.65.188:40053/ using document root <none> and password file <none>
I20260812 06:37:08.227144  1286 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:08.227193  1286 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:08.227313  1286 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:08.227666  1286 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786516597789727-1286-0/minicluster-data/master-2-root/instance:
uuid: "b99772be6e1f4daf80611e5c5833a7b1"
format_stamp: "Formatted at 2026-08-12 06:37:08 on dist-test-slave-92m1"
I20260812 06:37:08.229414  1286 fs_manager.cc:696] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20260812 06:37:08.229879  3164 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.65.189" port: 45799 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:08.230684  3177 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:08.230958  1286 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20260812 06:37:08.231060  1286 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786516597789727-1286-0/minicluster-data/master-2-root
uuid: "b99772be6e1f4daf80611e5c5833a7b1"
format_stamp: "Formatted at 2026-08-12 06:37:08 on dist-test-slave-92m1"
I20260812 06:37:08.231125  1286 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786516597789727-1286-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786516597789727-1286-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786516597789727-1286-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:08.234105  3164 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.65.188" port: 46273 } has no permanent_uuid. Determining permanent_uuid...
W20260812 06:37:08.235344  3164 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.1.65.188:46273: Network error: Client connection negotiation failed: client connection to 127.1.65.188:46273: connect: Connection refused (error 111)
I20260812 06:37:08.236559  3032 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.65.189" port: 45799 } attempt: 1
I20260812 06:37:08.240145  3032 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.65.188" port: 46273 } has no permanent_uuid. Determining permanent_uuid...
W20260812 06:37:08.241313  3032 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.1.65.188:46273: Network error: Client connection negotiation failed: client connection to 127.1.65.188:46273: connect: Connection refused (error 111)
I20260812 06:37:08.244292  1286 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:08.244609  1286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:08.249188  1286 rpc_server.cc:307] RPC server started. Bound to: 127.1.65.188:46273
I20260812 06:37:08.249294  3277 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.65.188:46273 every 8 connection(s)
I20260812 06:37:08.249616  1286 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260812 06:37:08.250015  3279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:08.250996  3279 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43845 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:08.254953  3279 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.65.189" port: 45799 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:08.258461  3279 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.65.188" port: 46273 } has no permanent_uuid. Determining permanent_uuid...
I20260812 06:37:08.263163  3279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b99772be6e1f4daf80611e5c5833a7b1: Bootstrap starting.
I20260812 06:37:08.264129  3279 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b99772be6e1f4daf80611e5c5833a7b1: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:08.265197  3279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b99772be6e1f4daf80611e5c5833a7b1: No bootstrap required, opened a new log
I20260812 06:37:08.265635  3279 raft_consensus.cc:359] T 00000000000000000000000000000000 P b99772be6e1f4daf80611e5c5833a7b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f281bbbdadfb4e87bb58276193f5be13" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43845 } } peers { permanent_uuid: "c6d40a4a3831497cbff775c15b18a6f0" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 45799 } } peers { permanent_uuid: "b99772be6e1f4daf80611e5c5833a7b1" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 46273 } }
I20260812 06:37:08.265743  3279 raft_consensus.cc:385] T 00000000000000000000000000000000 P b99772be6e1f4daf80611e5c5833a7b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:08.265791  3279 raft_consensus.cc:740] T 00000000000000000000000000000000 P b99772be6e1f4daf80611e5c5833a7b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b99772be6e1f4daf80611e5c5833a7b1, State: Initialized, Role: FOLLOWER
I20260812 06:37:08.265949  3279 consensus_queue.cc:260] T 00000000000000000000000000000000 P b99772be6e1f4daf80611e5c5833a7b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f281bbbdadfb4e87bb58276193f5be13" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43845 } } peers { permanent_uuid: "c6d40a4a3831497cbff775c15b18a6f0" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 45799 } } peers { permanent_uuid: "b99772be6e1f4daf80611e5c5833a7b1" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 46273 } }
I20260812 06:37:08.266250  3286 sys_catalog.cc:455] T 00000000000000000000000000000000 P b99772be6e1f4daf80611e5c5833a7b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f281bbbdadfb4e87bb58276193f5be13" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43845 } } peers { permanent_uuid: "c6d40a4a3831497cbff775c15b18a6f0" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 45799 } } peers { permanent_uuid: "b99772be6e1f4daf80611e5c5833a7b1" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 46273 } } }
I20260812 06:37:08.266330  3286 sys_catalog.cc:458] T 00000000000000000000000000000000 P b99772be6e1f4daf80611e5c5833a7b1 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:08.266456  3279 sys_catalog.cc:565] T 00000000000000000000000000000000 P b99772be6e1f4daf80611e5c5833a7b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:37:08.266510  3032 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.65.188" port: 46273 } attempt: 1
W20260812 06:37:08.269164  3304 catalog_manager.cc:1594] T 00000000000000000000000000000000 P b99772be6e1f4daf80611e5c5833a7b1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:37:08.269240  3304 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:37:08.271073  3032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f281bbbdadfb4e87bb58276193f5be13: Bootstrap starting.
I20260812 06:37:08.271968  3032 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f281bbbdadfb4e87bb58276193f5be13: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:08.272966  3032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f281bbbdadfb4e87bb58276193f5be13: No bootstrap required, opened a new log
I20260812 06:37:08.273382  3032 raft_consensus.cc:359] T 00000000000000000000000000000000 P f281bbbdadfb4e87bb58276193f5be13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f281bbbdadfb4e87bb58276193f5be13" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43845 } } peers { permanent_uuid: "c6d40a4a3831497cbff775c15b18a6f0" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 45799 } } peers { permanent_uuid: "b99772be6e1f4daf80611e5c5833a7b1" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 46273 } }
I20260812 06:37:08.273489  3032 raft_consensus.cc:385] T 00000000000000000000000000000000 P f281bbbdadfb4e87bb58276193f5be13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:08.273574  3032 raft_consensus.cc:740] T 00000000000000000000000000000000 P f281bbbdadfb4e87bb58276193f5be13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f281bbbdadfb4e87bb58276193f5be13, State: Initialized, Role: FOLLOWER
I20260812 06:37:08.273736  3032 consensus_queue.cc:260] T 00000000000000000000000000000000 P f281bbbdadfb4e87bb58276193f5be13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f281bbbdadfb4e87bb58276193f5be13" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43845 } } peers { permanent_uuid: "c6d40a4a3831497cbff775c15b18a6f0" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 45799 } } peers { permanent_uuid: "b99772be6e1f4daf80611e5c5833a7b1" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 46273 } }
I20260812 06:37:08.274045  3307 sys_catalog.cc:455] T 00000000000000000000000000000000 P f281bbbdadfb4e87bb58276193f5be13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f281bbbdadfb4e87bb58276193f5be13" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43845 } } peers { permanent_uuid: "c6d40a4a3831497cbff775c15b18a6f0" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 45799 } } peers { permanent_uuid: "b99772be6e1f4daf80611e5c5833a7b1" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 46273 } } }
I20260812 06:37:08.274128  3307 sys_catalog.cc:458] T 00000000000000000000000000000000 P f281bbbdadfb4e87bb58276193f5be13 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:08.274225  3032 sys_catalog.cc:565] T 00000000000000000000000000000000 P f281bbbdadfb4e87bb58276193f5be13 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:37:08.275875  1286 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
W20260812 06:37:08.276319  3325 catalog_manager.cc:1594] T 00000000000000000000000000000000 P f281bbbdadfb4e87bb58276193f5be13: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:37:08.276396  3325 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 06:37:08.291496  3164 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.65.188" port: 46273 } attempt: 1
I20260812 06:37:08.296135  3164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0: Bootstrap starting.
I20260812 06:37:08.297003  3164 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:08.298022  3164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0: No bootstrap required, opened a new log
I20260812 06:37:08.298437  3164 raft_consensus.cc:359] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f281bbbdadfb4e87bb58276193f5be13" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43845 } } peers { permanent_uuid: "c6d40a4a3831497cbff775c15b18a6f0" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 45799 } } peers { permanent_uuid: "b99772be6e1f4daf80611e5c5833a7b1" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 46273 } }
I20260812 06:37:08.298545  3164 raft_consensus.cc:385] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:08.298617  3164 raft_consensus.cc:740] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6d40a4a3831497cbff775c15b18a6f0, State: Initialized, Role: FOLLOWER
I20260812 06:37:08.298777  3164 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f281bbbdadfb4e87bb58276193f5be13" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43845 } } peers { permanent_uuid: "c6d40a4a3831497cbff775c15b18a6f0" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 45799 } } peers { permanent_uuid: "b99772be6e1f4daf80611e5c5833a7b1" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 46273 } }
I20260812 06:37:08.299108  3328 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f281bbbdadfb4e87bb58276193f5be13" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43845 } } peers { permanent_uuid: "c6d40a4a3831497cbff775c15b18a6f0" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 45799 } } peers { permanent_uuid: "b99772be6e1f4daf80611e5c5833a7b1" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 46273 } } }
I20260812 06:37:08.299188  3328 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:08.299288  3164 sys_catalog.cc:565] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:37:08.301014  1286 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20260812 06:37:08.301489  1286 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:08.301626  3347 catalog_manager.cc:1594] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 06:37:08.301684  3347 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260812 06:37:08.303741  3348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:08.303741  3349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:08.303809  3351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:08.304015  1286 server_base.cc:1061] running on GCE node
I20260812 06:37:08.304275  1286 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 06:37:08.304322  1286 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 06:37:08.304338  1286 hybrid_clock.cc:648] HybridClock initialized: now 1786516628304338 us; error 0 us; skew 500 ppm
I20260812 06:37:08.305243  1286 webserver.cc:533] Webserver started at http://127.1.65.129:43697/ using document root <none> and password file <none>
I20260812 06:37:08.305411  1286 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:08.305474  1286 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:08.305570  1286 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:08.305927  1286 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786516597789727-1286-0/minicluster-data/ts-0-root/instance:
uuid: "7b2c233dac474995aec8c2ccfb580090"
format_stamp: "Formatted at 2026-08-12 06:37:08 on dist-test-slave-92m1"
I20260812 06:37:08.307349  1286 fs_manager.cc:696] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.001s
I20260812 06:37:08.308207  3362 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:08.308420  1286 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20260812 06:37:08.308508  1286 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786516597789727-1286-0/minicluster-data/ts-0-root
uuid: "7b2c233dac474995aec8c2ccfb580090"
format_stamp: "Formatted at 2026-08-12 06:37:08 on dist-test-slave-92m1"
I20260812 06:37:08.308573  1286 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786516597789727-1286-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786516597789727-1286-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786516597789727-1286-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:08.351825  1286 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:08.352310  1286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:08.352658  1286 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:08.353183  1286 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:08.353245  1286 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:08.353307  1286 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:08.353354  1286 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:08.358479  1286 rpc_server.cc:307] RPC server started. Bound to: 127.1.65.129:44601
I20260812 06:37:08.358537  3476 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.65.129:44601 every 8 connection(s)
I20260812 06:37:08.363708  3485 heartbeater.cc:344] Connected to a master server at 127.1.65.189:45799
I20260812 06:37:08.363806  3485 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:08.363808  3479 heartbeater.cc:344] Connected to a master server at 127.1.65.190:43845
I20260812 06:37:08.363950  3479 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:08.364023  3485 heartbeater.cc:507] Master 127.1.65.189:45799 requested a full tablet report, sending...
I20260812 06:37:08.364153  3479 heartbeater.cc:507] Master 127.1.65.190:43845 requested a full tablet report, sending...
I20260812 06:37:08.364459  3477 heartbeater.cc:344] Connected to a master server at 127.1.65.188:46273
I20260812 06:37:08.364538  3477 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:08.364702  3477 heartbeater.cc:507] Master 127.1.65.188:46273 requested a full tablet report, sending...
I20260812 06:37:08.364748  3089 ts_manager.cc:194] Registered new tserver with Master: 7b2c233dac474995aec8c2ccfb580090 (127.1.65.129:44601)
I20260812 06:37:08.364830  2977 ts_manager.cc:194] Registered new tserver with Master: 7b2c233dac474995aec8c2ccfb580090 (127.1.65.129:44601)
I20260812 06:37:08.365377  3210 ts_manager.cc:194] Registered new tserver with Master: 7b2c233dac474995aec8c2ccfb580090 (127.1.65.129:44601)
I20260812 06:37:08.366032  1286 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.006966646s
I20260812 06:37:08.366554  1286 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 3.8968e-05s
I20260812 06:37:08.407351  3328 raft_consensus.cc:493] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:37:08.407591  3328 raft_consensus.cc:515] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f281bbbdadfb4e87bb58276193f5be13" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43845 } } peers { permanent_uuid: "c6d40a4a3831497cbff775c15b18a6f0" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 45799 } } peers { permanent_uuid: "b99772be6e1f4daf80611e5c5833a7b1" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 46273 } }
I20260812 06:37:08.408260  3328 leader_election.cc:290] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f281bbbdadfb4e87bb58276193f5be13 (127.1.65.190:43845), b99772be6e1f4daf80611e5c5833a7b1 (127.1.65.188:46273)
I20260812 06:37:08.409051  2990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c6d40a4a3831497cbff775c15b18a6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f281bbbdadfb4e87bb58276193f5be13" is_pre_election: true
I20260812 06:37:08.409096  3231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c6d40a4a3831497cbff775c15b18a6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b99772be6e1f4daf80611e5c5833a7b1" is_pre_election: true
I20260812 06:37:08.409250  2990 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f281bbbdadfb4e87bb58276193f5be13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6d40a4a3831497cbff775c15b18a6f0 in term 0.
I20260812 06:37:08.409261  3231 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b99772be6e1f4daf80611e5c5833a7b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6d40a4a3831497cbff775c15b18a6f0 in term 0.
I20260812 06:37:08.409909  3061 leader_election.cc:304] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6d40a4a3831497cbff775c15b18a6f0, f281bbbdadfb4e87bb58276193f5be13; no voters: 
I20260812 06:37:08.410385  3328 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:37:08.410441  3328 raft_consensus.cc:493] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:37:08.410476  3328 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:08.412024  3328 raft_consensus.cc:515] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f281bbbdadfb4e87bb58276193f5be13" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43845 } } peers { permanent_uuid: "c6d40a4a3831497cbff775c15b18a6f0" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 45799 } } peers { permanent_uuid: "b99772be6e1f4daf80611e5c5833a7b1" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 46273 } }
I20260812 06:37:08.413138  3231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c6d40a4a3831497cbff775c15b18a6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b99772be6e1f4daf80611e5c5833a7b1"
I20260812 06:37:08.413269  3231 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b99772be6e1f4daf80611e5c5833a7b1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:08.413254  2990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c6d40a4a3831497cbff775c15b18a6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f281bbbdadfb4e87bb58276193f5be13"
I20260812 06:37:08.413424  2990 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f281bbbdadfb4e87bb58276193f5be13 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:08.414476  3231 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b99772be6e1f4daf80611e5c5833a7b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6d40a4a3831497cbff775c15b18a6f0 in term 1.
I20260812 06:37:08.415017  2990 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f281bbbdadfb4e87bb58276193f5be13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6d40a4a3831497cbff775c15b18a6f0 in term 1.
I20260812 06:37:08.415409  3328 leader_election.cc:290] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [CANDIDATE]: Term 1 election: Requested vote from peers f281bbbdadfb4e87bb58276193f5be13 (127.1.65.190:43845), b99772be6e1f4daf80611e5c5833a7b1 (127.1.65.188:46273)
I20260812 06:37:08.415518  3061 leader_election.cc:304] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6d40a4a3831497cbff775c15b18a6f0, f281bbbdadfb4e87bb58276193f5be13; no voters: 
I20260812 06:37:08.415941  3328 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:37:08.416271  3328 raft_consensus.cc:697] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [term 1 LEADER]: Becoming Leader. State: Replica: c6d40a4a3831497cbff775c15b18a6f0, State: Running, Role: LEADER
I20260812 06:37:08.416535  3328 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f281bbbdadfb4e87bb58276193f5be13" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43845 } } peers { permanent_uuid: "c6d40a4a3831497cbff775c15b18a6f0" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 45799 } } peers { permanent_uuid: "b99772be6e1f4daf80611e5c5833a7b1" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 46273 } }
I20260812 06:37:08.418013  3514 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6d40a4a3831497cbff775c15b18a6f0. Latest consensus state: current_term: 1 leader_uuid: "c6d40a4a3831497cbff775c15b18a6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f281bbbdadfb4e87bb58276193f5be13" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43845 } } peers { permanent_uuid: "c6d40a4a3831497cbff775c15b18a6f0" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 45799 } } peers { permanent_uuid: "b99772be6e1f4daf80611e5c5833a7b1" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 46273 } } }
I20260812 06:37:08.418133  3514 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:08.418542  3519 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:37:08.419991  3519 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:37:08.423543  3231 raft_consensus.cc:1275] T 00000000000000000000000000000000 P b99772be6e1f4daf80611e5c5833a7b1 [term 1 FOLLOWER]: Refusing update from remote peer c6d40a4a3831497cbff775c15b18a6f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:37:08.423979  2990 raft_consensus.cc:1275] T 00000000000000000000000000000000 P f281bbbdadfb4e87bb58276193f5be13 [term 1 FOLLOWER]: Refusing update from remote peer c6d40a4a3831497cbff775c15b18a6f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:37:08.424492  3328 consensus_queue.cc:1048] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f281bbbdadfb4e87bb58276193f5be13" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:37:08.424765  3328 consensus_queue.cc:1048] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b99772be6e1f4daf80611e5c5833a7b1" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 46273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:37:08.426427  3286 sys_catalog.cc:455] T 00000000000000000000000000000000 P b99772be6e1f4daf80611e5c5833a7b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6d40a4a3831497cbff775c15b18a6f0. Latest consensus state: current_term: 1 leader_uuid: "c6d40a4a3831497cbff775c15b18a6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f281bbbdadfb4e87bb58276193f5be13" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43845 } } peers { permanent_uuid: "c6d40a4a3831497cbff775c15b18a6f0" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 45799 } } peers { permanent_uuid: "b99772be6e1f4daf80611e5c5833a7b1" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 46273 } } }
I20260812 06:37:08.426534  3286 sys_catalog.cc:458] T 00000000000000000000000000000000 P b99772be6e1f4daf80611e5c5833a7b1 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:08.426550  3307 sys_catalog.cc:455] T 00000000000000000000000000000000 P f281bbbdadfb4e87bb58276193f5be13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6d40a4a3831497cbff775c15b18a6f0. Latest consensus state: current_term: 1 leader_uuid: "c6d40a4a3831497cbff775c15b18a6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f281bbbdadfb4e87bb58276193f5be13" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43845 } } peers { permanent_uuid: "c6d40a4a3831497cbff775c15b18a6f0" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 45799 } } peers { permanent_uuid: "b99772be6e1f4daf80611e5c5833a7b1" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 46273 } } }
W20260812 06:37:08.426712  2957 tablet.cc:2406] T 00000000000000000000000000000000 P f281bbbdadfb4e87bb58276193f5be13: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:37:08.426728  3307 sys_catalog.cc:458] T 00000000000000000000000000000000 P f281bbbdadfb4e87bb58276193f5be13 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:08.428099  3328 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c6d40a4a3831497cbff775c15b18a6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f281bbbdadfb4e87bb58276193f5be13" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43845 } } peers { permanent_uuid: "c6d40a4a3831497cbff775c15b18a6f0" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 45799 } } peers { permanent_uuid: "b99772be6e1f4daf80611e5c5833a7b1" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 46273 } } }
I20260812 06:37:08.428210  3328 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:08.428316  3328 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c6d40a4a3831497cbff775c15b18a6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f281bbbdadfb4e87bb58276193f5be13" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43845 } } peers { permanent_uuid: "c6d40a4a3831497cbff775c15b18a6f0" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 45799 } } peers { permanent_uuid: "b99772be6e1f4daf80611e5c5833a7b1" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 46273 } } }
I20260812 06:37:08.428388  3328 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:08.428129  3286 sys_catalog.cc:455] T 00000000000000000000000000000000 P b99772be6e1f4daf80611e5c5833a7b1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c6d40a4a3831497cbff775c15b18a6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f281bbbdadfb4e87bb58276193f5be13" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43845 } } peers { permanent_uuid: "c6d40a4a3831497cbff775c15b18a6f0" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 45799 } } peers { permanent_uuid: "b99772be6e1f4daf80611e5c5833a7b1" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 46273 } } }
I20260812 06:37:08.428479  3286 sys_catalog.cc:458] T 00000000000000000000000000000000 P b99772be6e1f4daf80611e5c5833a7b1 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:08.429513  3307 sys_catalog.cc:455] T 00000000000000000000000000000000 P f281bbbdadfb4e87bb58276193f5be13 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c6d40a4a3831497cbff775c15b18a6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f281bbbdadfb4e87bb58276193f5be13" member_type: VOTER last_known_addr { host: "127.1.65.190" port: 43845 } } peers { permanent_uuid: "c6d40a4a3831497cbff775c15b18a6f0" member_type: VOTER last_known_addr { host: "127.1.65.189" port: 45799 } } peers { permanent_uuid: "b99772be6e1f4daf80611e5c5833a7b1" member_type: VOTER last_known_addr { host: "127.1.65.188" port: 46273 } } }
I20260812 06:37:08.429615  3307 sys_catalog.cc:458] T 00000000000000000000000000000000 P f281bbbdadfb4e87bb58276193f5be13 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 06:37:08.430624  3519 catalog_manager.cc:1383] Generated new cluster ID: 081b5869a6a6475d95deda71c973f7f6
I20260812 06:37:08.430704  3519 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
time=2026-08-12T06:37:08.439Z level=INFO source=main.go:1678 msg="updated GOGC" old=100 new=75
time=2026-08-12T06:37:08.447Z level=INFO source=main.go:744 msg="Leaving GOMAXPROCS=4: CPU quota undefined" component=automaxprocs
time=2026-08-12T06:37:08.448Z level=INFO source=memlimit.go:198 msg="GOMEMLIMIT is updated" component=automemlimit package=github.com/KimMachineGun/automemlimit/memlimit GOMEMLIMIT=14142552883 previous=9223372036854775807
time=2026-08-12T06:37:08.448Z level=INFO source=main.go:851 msg="Starting Prometheus Server" mode=server version="(version=3.11.2, branch=HEAD, revision=f0f0fdd679dcd6df320b0558b20919f7cd44c407)"
time=2026-08-12T06:37:08.448Z level=INFO source=main.go:856 msg="operational information" build_context="(go=go1.26.2, platform=linux/amd64, user=root@d2684485c347, date=20260413-11:53:40, tags=netgo,builtinassets)" host_details="(Linux 5.4.0-1157-gcp #166~18.04.1-Ubuntu SMP Wed Jan 21 01:57:46 UTC 2026 x86_64 dist-test-slave-92m1 (none))" fd_limits="(soft=1048576, hard=1048576)" vm_limits="(soft=unlimited, hard=unlimited)"
I20260812 06:37:08.451886  3519 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 06:37:08.453063  3519 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:37:08.467888  3519 catalog_manager.cc:6092] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0: Generated new TSK 0
I20260812 06:37:08.468066  3519 catalog_manager.cc:1550] Initializing in-progress tserver states...
time=2026-08-12T06:37:08.690Z level=INFO source=web.go:710 msg="Start listening for connections" component=web address=127.0.0.1:0
time=2026-08-12T06:37:08.691Z level=INFO source=main.go:1410 msg="Starting TSDB ..."
time=2026-08-12T06:37:08.694Z level=INFO source=tls_config.go:354 msg="Listening on" component=web address=127.0.0.1:34325
time=2026-08-12T06:37:08.694Z level=INFO source=tls_config.go:357 msg="TLS is disabled." component=web http2=false address=127.0.0.1:34325
time=2026-08-12T06:37:08.699Z level=INFO source=head.go:698 msg="Replaying on-disk memory mappable chunks if any" component=tsdb
time=2026-08-12T06:37:08.699Z level=INFO source=head.go:784 msg="On-disk memory mappable chunks replay completed" component=tsdb duration=2.764µs
time=2026-08-12T06:37:08.699Z level=INFO source=head.go:792 msg="Replaying WAL, this may take a while" component=tsdb
time=2026-08-12T06:37:08.700Z level=INFO source=head.go:865 msg="WAL segment loaded" component=tsdb segment=0 maxSegment=0 duration=503.624µs
time=2026-08-12T06:37:08.700Z level=INFO source=head.go:902 msg="WAL replay completed" component=tsdb checkpoint_replay_duration=95.838µs wal_replay_duration=580.26µs wbl_replay_duration=155ns chunk_snapshot_load_duration=0s mmap_chunk_replay_duration=2.764µs total_replay_duration=745.884µs
time=2026-08-12T06:37:08.707Z level=INFO source=main.go:1431 msg="filesystem information" fs_type=OVERLAYFS_SUPER_MAGIC
time=2026-08-12T06:37:08.707Z level=INFO source=main.go:1434 msg="TSDB started"
time=2026-08-12T06:37:08.707Z level=INFO source=main.go:1632 msg="Loading configuration file" filename=/tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786516597789727-1286-0/prometheus.yml
time=2026-08-12T06:37:08.707Z level=INFO source=main.go:1048 msg="TSDB retention updated" duration=15d size=0B percentage=0
time=2026-08-12T06:37:08.712Z level=INFO source=main.go:1671 msg="Completed loading of configuration file" db_storage=71.371µs remote_storage=2.266µs web_handler=636ns query_engine=1.076µs scrape=4.21223ms scrape_sd=149.359µs notify=1.649µs notify_sd=1.138µs rules=1.519µs tracing=3.85µs filename=/tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786516597789727-1286-0/prometheus.yml totalDuration=4.818777ms
time=2026-08-12T06:37:08.712Z level=INFO source=main.go:1395 msg="Server is ready to receive web requests."
time=2026-08-12T06:37:08.712Z level=INFO source=manager.go:209 msg="Starting rule manager..." component="rule manager"
I20260812 06:37:09.270265  3304 catalog_manager.cc:1295] Loaded cluster ID: 081b5869a6a6475d95deda71c973f7f6
I20260812 06:37:09.270368  3304 catalog_manager.cc:1588] T 00000000000000000000000000000000 P b99772be6e1f4daf80611e5c5833a7b1: loading cluster ID for follower catalog manager: success
I20260812 06:37:09.271643  3304 catalog_manager.cc:1610] T 00000000000000000000000000000000 P b99772be6e1f4daf80611e5c5833a7b1: acquiring CA information for follower catalog manager: success
I20260812 06:37:09.272197  3304 catalog_manager.cc:1638] T 00000000000000000000000000000000 P b99772be6e1f4daf80611e5c5833a7b1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:37:09.276983  3325 catalog_manager.cc:1295] Loaded cluster ID: 081b5869a6a6475d95deda71c973f7f6
I20260812 06:37:09.277031  3325 catalog_manager.cc:1588] T 00000000000000000000000000000000 P f281bbbdadfb4e87bb58276193f5be13: loading cluster ID for follower catalog manager: success
I20260812 06:37:09.278143  3325 catalog_manager.cc:1610] T 00000000000000000000000000000000 P f281bbbdadfb4e87bb58276193f5be13: acquiring CA information for follower catalog manager: success
I20260812 06:37:09.278678  3325 catalog_manager.cc:1638] T 00000000000000000000000000000000 P f281bbbdadfb4e87bb58276193f5be13: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:37:09.367328  3089 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47474
I20260812 06:37:10.368263  3485 heartbeater.cc:499] Master 127.1.65.189:45799 was elected leader, sending a full tablet report...
I20260812 06:37:16.304418  3347 catalog_manager.cc:1638] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 06:37:22.384047  3485 heartbeater.cc:499] Master 127.1.65.189:45799 was elected leader, sending a full tablet report...
time=2026-08-12T06:37:25.692Z level=WARN source=main.go:1191 msg="Received an OS signal, exiting gracefully..." signal=terminated
time=2026-08-12T06:37:25.692Z level=INFO source=main.go:1216 msg="Stopping scrape discovery manager..."
time=2026-08-12T06:37:25.692Z level=INFO source=main.go:1230 msg="Stopping notify discovery manager..."
time=2026-08-12T06:37:25.692Z level=INFO source=main.go:1245 msg="Stopping rule manager manager..."
time=2026-08-12T06:37:25.692Z level=INFO source=manager.go:223 msg="Stopping rule manager..." component="rule manager"
time=2026-08-12T06:37:25.692Z level=INFO source=manager.go:239 msg="Rule manager stopped" component="rule manager"
time=2026-08-12T06:37:25.692Z level=INFO source=main.go:1269 msg="Stopping scrape manager..."
time=2026-08-12T06:37:25.695Z level=INFO source=main.go:1241 msg="Rule manager stopped"
time=2026-08-12T06:37:25.695Z level=INFO source=main.go:1212 msg="Scrape discovery manager stopped"
time=2026-08-12T06:37:25.695Z level=INFO source=main.go:1261 msg="Scrape manager stopped"
time=2026-08-12T06:37:25.695Z level=INFO source=main.go:1226 msg="Notify discovery manager stopped"
time=2026-08-12T06:37:25.695Z level=INFO source=main.go:1284 msg="Stopping tracing manager..."
time=2026-08-12T06:37:25.695Z level=INFO source=main.go:1369 msg="Stopping reloaders..."
time=2026-08-12T06:37:25.695Z level=INFO source=main.go:1458 msg="Stopping storage..."
time=2026-08-12T06:37:25.695Z level=INFO source=main.go:1361 msg="Reloaders stopped"
time=2026-08-12T06:37:25.695Z level=INFO source=main.go:1280 msg="Tracing manager stopped"
time=2026-08-12T06:37:25.700Z level=INFO source=main.go:1536 msg="Stopping web handler..."
time=2026-08-12T06:37:25.700Z level=INFO source=main.go:1559 msg="Stopping notifier manager..."
time=2026-08-12T06:37:25.700Z level=INFO source=manager.go:321 msg="Stopping notification manager..." component=notifier
time=2026-08-12T06:37:25.700Z level=INFO source=main.go:1454 msg="TSDB stopped"
time=2026-08-12T06:37:25.700Z level=INFO source=main.go:1555 msg="Notifier manager stopped"
time=2026-08-12T06:37:25.700Z level=INFO source=main.go:1532 msg="Web handler stopped"
time=2026-08-12T06:37:25.701Z level=INFO source=main.go:1570 msg="See you next time!"
I20260812 06:37:25.713930  1286 tablet_server.cc:179] TabletServer@127.1.65.129:0 shutting down...
I20260812 06:37:25.719033  1286 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260812 06:37:25.731412  1286 tablet_server.cc:196] TabletServer@127.1.65.129:0 shutdown complete.
I20260812 06:37:25.733669  1286 master.cc:562] Master@127.1.65.190:43845 shutting down...
I20260812 06:37:25.737864  1286 raft_consensus.cc:2243] T 00000000000000000000000000000000 P f281bbbdadfb4e87bb58276193f5be13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:37:25.738025  1286 raft_consensus.cc:2272] T 00000000000000000000000000000000 P f281bbbdadfb4e87bb58276193f5be13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:37:25.738076  1286 tablet_replica.cc:333] T 00000000000000000000000000000000 P f281bbbdadfb4e87bb58276193f5be13: stopping tablet replica
I20260812 06:37:25.750862  1286 master.cc:584] Master@127.1.65.190:43845 shutdown complete.
I20260812 06:37:25.753365  1286 master.cc:562] Master@127.1.65.189:45799 shutting down...
I20260812 06:37:25.757413  1286 raft_consensus.cc:2243] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:37:25.757714  1286 raft_consensus.cc:2272] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:37:25.757812  1286 tablet_replica.cc:333] T 00000000000000000000000000000000 P c6d40a4a3831497cbff775c15b18a6f0: stopping tablet replica
I20260812 06:37:25.770463  1286 master.cc:584] Master@127.1.65.189:45799 shutdown complete.
I20260812 06:37:25.772975  1286 master.cc:562] Master@127.1.65.188:46273 shutting down...
I20260812 06:37:25.777220  1286 raft_consensus.cc:2243] T 00000000000000000000000000000000 P b99772be6e1f4daf80611e5c5833a7b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:37:25.777377  1286 raft_consensus.cc:2272] T 00000000000000000000000000000000 P b99772be6e1f4daf80611e5c5833a7b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:37:25.777443  1286 tablet_replica.cc:333] T 00000000000000000000000000000000 P b99772be6e1f4daf80611e5c5833a7b1: stopping tablet replica
I20260812 06:37:25.790561  1286 master.cc:584] Master@127.1.65.188:46273 shutdown complete.
[       OK ] MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower (17635 ms)
[----------] 2 tests from MultiMasterPrometheusSDITest (25292 ms total)

[----------] 1 test from LocationLabelSDITest
[ RUN      ] LocationLabelSDITest.TserverLocationLabel
I20260812 06:37:25.796684  1286 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.1.65.190:36219
I20260812 06:37:25.797029  1286 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:25.798942  3549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:25.798986  3552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:25.799026  3548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:25.799155  1286 server_base.cc:1061] running on GCE node
I20260812 06:37:25.799476  1286 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 06:37:25.799536  1286 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 06:37:25.799561  1286 hybrid_clock.cc:648] HybridClock initialized: now 1786516645799561 us; error 0 us; skew 500 ppm
I20260812 06:37:25.800390  1286 webserver.cc:533] Webserver started at http://127.1.65.190:34481/ using document root <none> and password file <none>
I20260812 06:37:25.800555  1286 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:25.800642  1286 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:25.800740  1286 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:25.801131  1286 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786516597789727-1286-0/minicluster-data/master-0-root/instance:
uuid: "216d144f7e7841e8bafdc1949abe69a2"
format_stamp: "Formatted at 2026-08-12 06:37:25 on dist-test-slave-92m1"
I20260812 06:37:25.802655  1286 fs_manager.cc:696] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20260812 06:37:25.803582  3560 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:25.803797  1286 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20260812 06:37:25.803889  1286 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786516597789727-1286-0/minicluster-data/master-0-root
uuid: "216d144f7e7841e8bafdc1949abe69a2"
format_stamp: "Formatted at 2026-08-12 06:37:25 on dist-test-slave-92m1"
I20260812 06:37:25.803973  1286 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786516597789727-1286-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786516597789727-1286-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786516597789727-1286-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:25.829643  1286 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:25.830116  1286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:25.834769  1286 rpc_server.cc:307] RPC server started. Bound to: 127.1.65.190:36219
I20260812 06:37:25.837363  3654 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.65.190:36219 every 8 connection(s)
I20260812 06:37:25.837975  3655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:25.839807  3655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 216d144f7e7841e8bafdc1949abe69a2: Bootstrap starting.
I20260812 06:37:25.840648  3655 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 216d144f7e7841e8bafdc1949abe69a2: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:25.841744  3655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 216d144f7e7841e8bafdc1949abe69a2: No bootstrap required, opened a new log
I20260812 06:37:25.842187  3655 raft_consensus.cc:359] T 00000000000000000000000000000000 P 216d144f7e7841e8bafdc1949abe69a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "216d144f7e7841e8bafdc1949abe69a2" member_type: VOTER }
I20260812 06:37:25.842296  3655 raft_consensus.cc:385] T 00000000000000000000000000000000 P 216d144f7e7841e8bafdc1949abe69a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:25.842348  3655 raft_consensus.cc:740] T 00000000000000000000000000000000 P 216d144f7e7841e8bafdc1949abe69a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 216d144f7e7841e8bafdc1949abe69a2, State: Initialized, Role: FOLLOWER
I20260812 06:37:25.842497  3655 consensus_queue.cc:260] T 00000000000000000000000000000000 P 216d144f7e7841e8bafdc1949abe69a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "216d144f7e7841e8bafdc1949abe69a2" member_type: VOTER }
I20260812 06:37:25.842587  3655 raft_consensus.cc:399] T 00000000000000000000000000000000 P 216d144f7e7841e8bafdc1949abe69a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:37:25.842631  3655 raft_consensus.cc:493] T 00000000000000000000000000000000 P 216d144f7e7841e8bafdc1949abe69a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:37:25.842686  3655 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 216d144f7e7841e8bafdc1949abe69a2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:25.843372  3655 raft_consensus.cc:515] T 00000000000000000000000000000000 P 216d144f7e7841e8bafdc1949abe69a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "216d144f7e7841e8bafdc1949abe69a2" member_type: VOTER }
I20260812 06:37:25.843521  3655 leader_election.cc:304] T 00000000000000000000000000000000 P 216d144f7e7841e8bafdc1949abe69a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 216d144f7e7841e8bafdc1949abe69a2; no voters: 
I20260812 06:37:25.843739  3655 leader_election.cc:290] T 00000000000000000000000000000000 P 216d144f7e7841e8bafdc1949abe69a2 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 06:37:25.843870  3660 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 216d144f7e7841e8bafdc1949abe69a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:37:25.844115  3660 raft_consensus.cc:697] T 00000000000000000000000000000000 P 216d144f7e7841e8bafdc1949abe69a2 [term 1 LEADER]: Becoming Leader. State: Replica: 216d144f7e7841e8bafdc1949abe69a2, State: Running, Role: LEADER
I20260812 06:37:25.844218  3655 sys_catalog.cc:565] T 00000000000000000000000000000000 P 216d144f7e7841e8bafdc1949abe69a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:37:25.844254  3660 consensus_queue.cc:237] T 00000000000000000000000000000000 P 216d144f7e7841e8bafdc1949abe69a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "216d144f7e7841e8bafdc1949abe69a2" member_type: VOTER }
I20260812 06:37:25.844792  3661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 216d144f7e7841e8bafdc1949abe69a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "216d144f7e7841e8bafdc1949abe69a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "216d144f7e7841e8bafdc1949abe69a2" member_type: VOTER } }
I20260812 06:37:25.844842  3668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 216d144f7e7841e8bafdc1949abe69a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 216d144f7e7841e8bafdc1949abe69a2. Latest consensus state: current_term: 1 leader_uuid: "216d144f7e7841e8bafdc1949abe69a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "216d144f7e7841e8bafdc1949abe69a2" member_type: VOTER } }
I20260812 06:37:25.844975  3668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 216d144f7e7841e8bafdc1949abe69a2 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:25.845253  3675 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:37:25.845438  3661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 216d144f7e7841e8bafdc1949abe69a2 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:25.846168  3675 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:37:25.846433  1286 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260812 06:37:25.848070  3675 catalog_manager.cc:1383] Generated new cluster ID: eae8ddca567d481f8da633a7d29f6da6
I20260812 06:37:25.848132  3675 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:37:25.854947  3675 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 06:37:25.855563  3675 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:37:25.862433  3675 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 216d144f7e7841e8bafdc1949abe69a2: Generated new TSK 0
I20260812 06:37:25.862610  3675 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 06:37:25.878764  1286 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:25.880918  3697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:25.880966  3698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:25.881023  1286 server_base.cc:1061] running on GCE node
W20260812 06:37:25.880966  3700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:25.881378  1286 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 06:37:25.881423  1286 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 06:37:25.881439  1286 hybrid_clock.cc:648] HybridClock initialized: now 1786516645881439 us; error 0 us; skew 500 ppm
I20260812 06:37:25.882261  1286 webserver.cc:533] Webserver started at http://127.1.65.129:37785/ using document root <none> and password file <none>
I20260812 06:37:25.882390  1286 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:25.882431  1286 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:25.882490  1286 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:25.882817  1286 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786516597789727-1286-0/minicluster-data/ts-0-root/instance:
uuid: "4b7dd6f41b63424486615ffc3b391c0f"
format_stamp: "Formatted at 2026-08-12 06:37:25 on dist-test-slave-92m1"
I20260812 06:37:25.884713  1286 fs_manager.cc:696] Time spent creating directory manager: real 0.002s	user 0.000s	sys 0.002s
I20260812 06:37:25.885661  3713 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:25.885921  1286 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20260812 06:37:25.886026  1286 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786516597789727-1286-0/minicluster-data/ts-0-root
uuid: "4b7dd6f41b63424486615ffc3b391c0f"
format_stamp: "Formatted at 2026-08-12 06:37:25 on dist-test-slave-92m1"
I20260812 06:37:25.886129  1286 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786516597789727-1286-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786516597789727-1286-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786516597789727-1286-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:25.902117  1286 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:25.902511  1286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:25.902814  1286 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:25.903327  1286 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:25.903386  1286 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:25.903445  1286 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:25.903496  1286 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:25.908290  1286 rpc_server.cc:307] RPC server started. Bound to: 127.1.65.129:42775
I20260812 06:37:25.908324  3815 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.65.129:42775 every 8 connection(s)
I20260812 06:37:25.909075  1286 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:25.911144  3823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:25.911190  3821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:25.911406  3827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:25.911477  1286 server_base.cc:1061] running on GCE node
I20260812 06:37:25.911717  1286 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 06:37:25.911751  1286 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 06:37:25.911765  1286 hybrid_clock.cc:648] HybridClock initialized: now 1786516645911766 us; error 0 us; skew 500 ppm
I20260812 06:37:25.912695  1286 webserver.cc:533] Webserver started at http://127.1.65.130:39827/ using document root <none> and password file <none>
I20260812 06:37:25.912874  1286 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:25.912916  1286 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:25.912998  1286 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:25.913383  1286 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786516597789727-1286-0/minicluster-data/ts-1-root/instance:
uuid: "4020521625e74ceab81896cd1670d796"
format_stamp: "Formatted at 2026-08-12 06:37:25 on dist-test-slave-92m1"
I20260812 06:37:25.914847  1286 fs_manager.cc:696] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20260812 06:37:25.915791  3836 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:25.916082  1286 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.000s
I20260812 06:37:25.916153  1286 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786516597789727-1286-0/minicluster-data/ts-1-root
uuid: "4020521625e74ceab81896cd1670d796"
format_stamp: "Formatted at 2026-08-12 06:37:25 on dist-test-slave-92m1"
I20260812 06:37:25.916241  1286 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786516597789727-1286-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786516597789727-1286-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786516597789727-1286-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:25.917856  3816 heartbeater.cc:344] Connected to a master server at 127.1.65.190:36219
I20260812 06:37:25.917970  3816 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:25.918210  3816 heartbeater.cc:507] Master 127.1.65.190:36219 requested a full tablet report, sending...
I20260812 06:37:25.947489  1286 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:25.947966  1286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:25.948354  1286 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:25.948870  1286 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:25.948909  1286 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:25.948943  1286 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:25.949003  1286 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:25.955551  1286 rpc_server.cc:307] RPC server started. Bound to: 127.1.65.130:32837
I20260812 06:37:25.955648  3955 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.1.65.130:32837 every 8 connection(s)
I20260812 06:37:25.960934  3957 heartbeater.cc:344] Connected to a master server at 127.1.65.190:36219
I20260812 06:37:25.961030  3957 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:25.961199  3957 heartbeater.cc:507] Master 127.1.65.190:36219 requested a full tablet report, sending...
I20260812 06:37:25.968643  3595 ts_manager.cc:194] Registered new tserver with Master: 4b7dd6f41b63424486615ffc3b391c0f (127.1.65.129:42775)
I20260812 06:37:25.969969  3595 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53718
I20260812 06:37:26.005399  3592 ts_manager.cc:194] Registered new tserver with Master: 4020521625e74ceab81896cd1670d796 (127.1.65.130:32837)
I20260812 06:37:26.005614  1286 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.049473415s
I20260812 06:37:26.006147  3592 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53722
I20260812 06:37:26.006362  1286 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 4.336e-05s
time=2026-08-12T06:37:26.075Z level=INFO source=main.go:1678 msg="updated GOGC" old=100 new=75
time=2026-08-12T06:37:26.078Z level=INFO source=main.go:744 msg="Leaving GOMAXPROCS=4: CPU quota undefined" component=automaxprocs
time=2026-08-12T06:37:26.079Z level=INFO source=memlimit.go:198 msg="GOMEMLIMIT is updated" component=automemlimit package=github.com/KimMachineGun/automemlimit/memlimit GOMEMLIMIT=14142552883 previous=9223372036854775807
time=2026-08-12T06:37:26.079Z level=INFO source=main.go:851 msg="Starting Prometheus Server" mode=server version="(version=3.11.2, branch=HEAD, revision=f0f0fdd679dcd6df320b0558b20919f7cd44c407)"
time=2026-08-12T06:37:26.079Z level=INFO source=main.go:856 msg="operational information" build_context="(go=go1.26.2, platform=linux/amd64, user=root@d2684485c347, date=20260413-11:53:40, tags=netgo,builtinassets)" host_details="(Linux 5.4.0-1157-gcp #166~18.04.1-Ubuntu SMP Wed Jan 21 01:57:46 UTC 2026 x86_64 dist-test-slave-92m1 (none))" fd_limits="(soft=1048576, hard=1048576)" vm_limits="(soft=unlimited, hard=unlimited)"
time=2026-08-12T06:37:26.341Z level=INFO source=web.go:710 msg="Start listening for connections" component=web address=127.0.0.1:0
time=2026-08-12T06:37:26.342Z level=INFO source=main.go:1410 msg="Starting TSDB ..."
time=2026-08-12T06:37:26.343Z level=INFO source=tls_config.go:354 msg="Listening on" component=web address=127.0.0.1:36741
time=2026-08-12T06:37:26.343Z level=INFO source=tls_config.go:357 msg="TLS is disabled." component=web http2=false address=127.0.0.1:36741
time=2026-08-12T06:37:26.350Z level=INFO source=head.go:698 msg="Replaying on-disk memory mappable chunks if any" component=tsdb
time=2026-08-12T06:37:26.350Z level=INFO source=head.go:784 msg="On-disk memory mappable chunks replay completed" component=tsdb duration=2.032µs
time=2026-08-12T06:37:26.350Z level=INFO source=head.go:792 msg="Replaying WAL, this may take a while" component=tsdb
time=2026-08-12T06:37:26.350Z level=INFO source=head.go:865 msg="WAL segment loaded" component=tsdb segment=0 maxSegment=0 duration=387.963µs
time=2026-08-12T06:37:26.350Z level=INFO source=head.go:902 msg="WAL replay completed" component=tsdb checkpoint_replay_duration=70.376µs wal_replay_duration=440.84µs wbl_replay_duration=161ns chunk_snapshot_load_duration=0s mmap_chunk_replay_duration=2.032µs total_replay_duration=567.797µs
time=2026-08-12T06:37:26.352Z level=INFO source=main.go:1431 msg="filesystem information" fs_type=OVERLAYFS_SUPER_MAGIC
time=2026-08-12T06:37:26.352Z level=INFO source=main.go:1434 msg="TSDB started"
time=2026-08-12T06:37:26.352Z level=INFO source=main.go:1632 msg="Loading configuration file" filename=/tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786516597789727-1286-0/prometheus.yml
time=2026-08-12T06:37:26.353Z level=INFO source=main.go:1048 msg="TSDB retention updated" duration=15d size=0B percentage=0
time=2026-08-12T06:37:26.354Z level=INFO source=main.go:1671 msg="Completed loading of configuration file" db_storage=88.705µs remote_storage=2.238µs web_handler=621ns query_engine=1.101µs scrape=1.300769ms scrape_sd=167.168µs notify=2.472µs notify_sd=1.56µs rules=4.133µs tracing=15.06µs filename=/tmp/dist-test-taskkqcCVR/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786516597789727-1286-0/prometheus.yml totalDuration=2.022159ms
time=2026-08-12T06:37:26.355Z level=INFO source=main.go:1395 msg="Server is ready to receive web requests."
time=2026-08-12T06:37:26.355Z level=INFO source=manager.go:209 msg="Starting rule manager..." component="rule manager"
I20260812 06:37:26.971554  3816 heartbeater.cc:499] Master 127.1.65.190:36219 was elected leader, sending a full tablet report...
I20260812 06:37:27.013692  3957 heartbeater.cc:499] Master 127.1.65.190:36219 was elected leader, sending a full tablet report...
time=2026-08-12T06:37:33.327Z level=WARN source=main.go:1191 msg="Received an OS signal, exiting gracefully..." signal=terminated
time=2026-08-12T06:37:33.327Z level=INFO source=main.go:1216 msg="Stopping scrape discovery manager..."
time=2026-08-12T06:37:33.327Z level=INFO source=main.go:1230 msg="Stopping notify discovery manager..."
time=2026-08-12T06:37:33.327Z level=INFO source=main.go:1245 msg="Stopping rule manager manager..."
time=2026-08-12T06:37:33.327Z level=INFO source=manager.go:223 msg="Stopping rule manager..." component="rule manager"
time=2026-08-12T06:37:33.327Z level=INFO source=main.go:1212 msg="Scrape discovery manager stopped"
time=2026-08-12T06:37:33.327Z level=INFO source=manager.go:239 msg="Rule manager stopped" component="rule manager"
time=2026-08-12T06:37:33.327Z level=INFO source=main.go:1269 msg="Stopping scrape manager..."
time=2026-08-12T06:37:33.327Z level=INFO source=main.go:1241 msg="Rule manager stopped"
time=2026-08-12T06:37:33.327Z level=INFO source=main.go:1284 msg="Stopping tracing manager..."
time=2026-08-12T06:37:33.327Z level=INFO source=main.go:1261 msg="Scrape manager stopped"
time=2026-08-12T06:37:33.327Z level=INFO source=main.go:1369 msg="Stopping reloaders..."
time=2026-08-12T06:37:33.328Z level=INFO source=main.go:1226 msg="Notify discovery manager stopped"
time=2026-08-12T06:37:33.327Z level=INFO source=main.go:1458 msg="Stopping storage..."
time=2026-08-12T06:37:33.328Z level=INFO source=main.go:1280 msg="Tracing manager stopped"
time=2026-08-12T06:37:33.328Z level=INFO source=main.go:1361 msg="Reloaders stopped"
time=2026-08-12T06:37:33.333Z level=INFO source=main.go:1536 msg="Stopping web handler..."
time=2026-08-12T06:37:33.333Z level=INFO source=main.go:1559 msg="Stopping notifier manager..."
time=2026-08-12T06:37:33.333Z level=INFO source=manager.go:321 msg="Stopping notification manager..." component=notifier
time=2026-08-12T06:37:33.333Z level=INFO source=main.go:1555 msg="Notifier manager stopped"
time=2026-08-12T06:37:33.333Z level=INFO source=main.go:1454 msg="TSDB stopped"
time=2026-08-12T06:37:33.333Z level=INFO source=main.go:1532 msg="Web handler stopped"
time=2026-08-12T06:37:33.334Z level=INFO source=main.go:1570 msg="See you next time!"
I20260812 06:37:33.348869  1286 tablet_server.cc:179] TabletServer@127.1.65.129:0 shutting down...
I20260812 06:37:33.354549  1286 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260812 06:37:33.367539  1286 tablet_server.cc:196] TabletServer@127.1.65.129:0 shutdown complete.
I20260812 06:37:33.370608  1286 tablet_server.cc:179] TabletServer@127.1.65.130:0 shutting down...
I20260812 06:37:33.377460  1286 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260812 06:37:33.390342  1286 tablet_server.cc:196] TabletServer@127.1.65.130:0 shutdown complete.
I20260812 06:37:33.393085  1286 master.cc:562] Master@127.1.65.190:36219 shutting down...
I20260812 06:37:33.397918  1286 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 216d144f7e7841e8bafdc1949abe69a2 [term 1 LEADER]: Raft consensus shutting down.
I20260812 06:37:33.398149  1286 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 216d144f7e7841e8bafdc1949abe69a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:37:33.398231  1286 tablet_replica.cc:333] T 00000000000000000000000000000000 P 216d144f7e7841e8bafdc1949abe69a2: stopping tablet replica
I20260812 06:37:33.411621  1286 master.cc:584] Master@127.1.65.190:36219 shutdown complete.
[       OK ] LocationLabelSDITest.TserverLocationLabel (7623 ms)
[----------] 1 test from LocationLabelSDITest (7623 ms total)

[----------] Global test environment tear-down
[==========] 6 tests from 3 test suites ran. (55621 ms total)
[  PASSED  ] 6 tests.
I20260812 06:37:33.420842  1286 logging.cc:436] LogThrottler /workspace/apache/dev/local/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 2 messages since previous log ~25 seconds ago
