[2026-04-15 22:41:05,467][  INFO][PID:2784953] Marking build as starting
[2026-04-15 22:41:05,534][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:41:05,536][  INFO][PID:2784953] VM allocation process starts
[2026-04-15 22:41:05,573][  INFO][PID:2784953] Trying to allocate VM: ResallocHost, ticket_id=3164, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:41:10,576][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:41:15,577][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:41:20,578][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:41:25,580][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:41:30,585][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:41:35,588][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:41:40,590][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:41:45,596][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:41:50,599][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:41:55,603][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:42:00,608][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:42:05,611][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:42:10,620][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:42:15,623][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:42:20,629][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:42:25,633][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:42:30,635][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:42:35,638][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:42:40,642][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:42:45,643][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:42:50,645][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:42:55,646][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:43:00,648][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:43:05,651][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:43:10,654][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:43:15,658][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:43:20,663][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:43:25,666][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:43:30,668][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:43:35,673][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:43:40,677][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:43:45,687][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:43:50,692][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:43:55,694][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:44:00,701][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:44:05,708][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:44:10,712][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:44:15,715][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:44:20,723][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:44:25,724][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:44:30,726][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:44:35,729][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:44:40,734][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:44:45,737][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:44:50,740][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:44:55,742][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:45:00,746][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:45:05,749][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:45:10,752][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:45:15,756][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:45:20,762][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:45:23,862][  INFO][PID:2784953] Allocated host ResallocHost, ticket_id=3164, hostname=140.211.168.169, name=copr_osuosl_p09_dev_02147484_20260415_224046, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:45:23,863][  INFO][PID:2784953] Allocating ssh connection to builder
[2026-04-15 22:45:23,863][  INFO][PID:2784953] Checking that builder machine is OK
[2026-04-15 22:45:25,379][  INFO][PID:2784953] Installed copr-rpmbuild version: 1.6
[2026-04-15 22:45:25,380][ ERROR][PID:2784953] Re-try request for task on 'ResallocHost, ticket_id=3164, hostname=140.211.168.169, name=copr_osuosl_p09_dev_02147484_20260415_224046, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1
[2026-04-15 22:45:25,382][  INFO][PID:2784953] VM Release request
[2026-04-15 22:45:25,407][  INFO][PID:2784953] Retry #1 (on other host)
[2026-04-15 22:45:25,411][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:45:25,415][  INFO][PID:2784953] VM allocation process starts
[2026-04-15 22:45:25,459][  INFO][PID:2784953] Trying to allocate VM: ResallocHost, ticket_id=3247, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:45:30,466][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:45:35,468][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:45:40,475][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:45:45,480][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:45:50,491][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:45:55,499][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:46:00,507][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:46:05,512][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:46:10,515][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:46:15,523][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:46:20,528][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:46:25,533][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:46:30,540][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:46:35,553][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:46:40,560][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:46:45,569][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:46:50,575][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:46:55,582][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:47:00,585][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:47:05,593][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:47:10,602][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:47:15,609][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:47:20,614][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:47:25,624][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:47:30,629][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:47:35,634][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:47:40,636][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:47:45,643][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:47:50,646][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:47:55,650][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:48:00,655][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:48:05,664][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:48:10,670][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:48:15,684][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:48:20,695][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:48:25,710][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:48:30,717][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:48:35,726][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:48:40,739][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:48:43,993][  INFO][PID:2784953] Allocated host ResallocHost, ticket_id=3247, hostname=2620:52:6:1161:dead:beef:cafe:c906, name=vmhost_x86_02_dev_02147501_20260415_224505, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:48:43,995][  INFO][PID:2784953] Allocating ssh connection to builder
[2026-04-15 22:48:43,998][  INFO][PID:2784953] Checking that builder machine is OK
[2026-04-15 22:48:44,659][  INFO][PID:2784953] Installed copr-rpmbuild version: 1.6
[2026-04-15 22:48:44,660][ ERROR][PID:2784953] Re-try request for task on 'ResallocHost, ticket_id=3247, hostname=2620:52:6:1161:dead:beef:cafe:c906, name=vmhost_x86_02_dev_02147501_20260415_224505, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1
[2026-04-15 22:48:44,663][  INFO][PID:2784953] VM Release request
[2026-04-15 22:48:44,707][  INFO][PID:2784953] Retry #2 (on other host)
[2026-04-15 22:48:44,710][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:48:44,713][  INFO][PID:2784953] VM allocation process starts
[2026-04-15 22:48:44,814][  INFO][PID:2784953] Trying to allocate VM: ResallocHost, ticket_id=3274, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:48:49,828][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:48:50,982][  INFO][PID:2784953] Allocated host ResallocHost, ticket_id=3274, hostname=2620:52:6:1161:dead:beef:cafe:c906, name=vmhost_x86_02_dev_02147501_20260415_224505, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:48:50,983][  INFO][PID:2784953] Allocating ssh connection to builder
[2026-04-15 22:48:50,985][  INFO][PID:2784953] Checking that builder machine is OK
[2026-04-15 22:48:51,101][  INFO][PID:2784953] Installed copr-rpmbuild version: 1.6
[2026-04-15 22:48:51,103][ ERROR][PID:2784953] Re-try request for task on 'ResallocHost, ticket_id=3274, hostname=2620:52:6:1161:dead:beef:cafe:c906, name=vmhost_x86_02_dev_02147501_20260415_224505, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1
[2026-04-15 22:48:51,106][  INFO][PID:2784953] VM Release request
[2026-04-15 22:48:51,139][  INFO][PID:2784953] Retry #3 (on other host)
[2026-04-15 22:48:51,141][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:48:51,148][  INFO][PID:2784953] VM allocation process starts
[2026-04-15 22:48:51,189][  INFO][PID:2784953] Trying to allocate VM: ResallocHost, ticket_id=3275, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:48:54,267][  INFO][PID:2784953] Allocated host ResallocHost, ticket_id=3275, hostname=2620:52:6:1161:dead:beef:cafe:c906, name=vmhost_x86_02_dev_02147501_20260415_224505, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:48:54,268][  INFO][PID:2784953] Allocating ssh connection to builder
[2026-04-15 22:48:54,273][  INFO][PID:2784953] Checking that builder machine is OK
[2026-04-15 22:48:54,389][  INFO][PID:2784953] Installed copr-rpmbuild version: 1.6
[2026-04-15 22:48:54,390][ ERROR][PID:2784953] Re-try request for task on 'ResallocHost, ticket_id=3275, hostname=2620:52:6:1161:dead:beef:cafe:c906, name=vmhost_x86_02_dev_02147501_20260415_224505, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1
[2026-04-15 22:48:54,392][  INFO][PID:2784953] VM Release request
[2026-04-15 22:48:54,424][  INFO][PID:2784953] Retry #4 (on other host)
[2026-04-15 22:48:54,426][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:48:54,430][  INFO][PID:2784953] VM allocation process starts
[2026-04-15 22:48:54,480][  INFO][PID:2784953] Trying to allocate VM: ResallocHost, ticket_id=3276, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:48:59,487][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:49:00,662][  INFO][PID:2784953] Allocated host ResallocHost, ticket_id=3276, hostname=2620:52:6:1161:dead:beef:cafe:c906, name=vmhost_x86_02_dev_02147501_20260415_224505, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:49:00,666][  INFO][PID:2784953] Allocating ssh connection to builder
[2026-04-15 22:49:00,668][  INFO][PID:2784953] Checking that builder machine is OK
[2026-04-15 22:49:00,814][  INFO][PID:2784953] Installed copr-rpmbuild version: 1.6
[2026-04-15 22:49:00,815][ ERROR][PID:2784953] Re-try request for task on 'ResallocHost, ticket_id=3276, hostname=2620:52:6:1161:dead:beef:cafe:c906, name=vmhost_x86_02_dev_02147501_20260415_224505, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1
[2026-04-15 22:49:00,817][  INFO][PID:2784953] VM Release request
[2026-04-15 22:49:00,883][  INFO][PID:2784953] Retry #5 (on other host)
[2026-04-15 22:49:00,892][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:49:00,899][  INFO][PID:2784953] VM allocation process starts
[2026-04-15 22:49:00,971][  INFO][PID:2784953] Trying to allocate VM: ResallocHost, ticket_id=3277, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:49:05,996][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:49:11,001][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:49:16,015][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:49:21,030][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:49:26,041][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:49:31,043][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:49:36,049][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:49:41,051][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:49:46,053][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:49:51,054][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:49:56,056][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:50:01,060][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:50:06,064][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:50:11,073][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:50:16,079][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:50:21,085][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:50:26,090][  INFO][PID:2784953] Checking for cancel request
[2026-04-15 22:50:29,300][  INFO][PID:2784953] Allocated host ResallocHost, ticket_id=3277, hostname=2620:52:6:1161:dead:beef:cafe:c90a, name=vmhost_x86_02_dev_02147516_20260415_224749, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:50:29,301][  INFO][PID:2784953] Allocating ssh connection to builder
[2026-04-15 22:50:29,303][  INFO][PID:2784953] Checking that builder machine is OK
[2026-04-15 22:50:29,927][  INFO][PID:2784953] Installed copr-rpmbuild version: 1.6
[2026-04-15 22:50:29,928][ ERROR][PID:2784953] Re-try request for task on 'ResallocHost, ticket_id=3277, hostname=2620:52:6:1161:dead:beef:cafe:c90a, name=vmhost_x86_02_dev_02147516_20260415_224749, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1
[2026-04-15 22:50:29,928][  INFO][PID:2784953] VM Release request
[2026-04-15 22:50:29,946][ ERROR][PID:2784953] Backend process error: Three host tried without success: {'2620:52:6:1161:dead:beef:cafe:c90a', '140.211.168.169', '2620:52:6:1161:dead:beef:cafe:c906'}
[2026-04-15 22:50:29,947][WARNING][PID:2784953] Switching not-finished job state to 'failed'
[2026-04-15 22:50:29,948][  INFO][PID:2784953] Worker failed build, took 564.4859714508057
[2026-04-15 22:50:29,948][  INFO][PID:2784953] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 108000,
            "frontend_base_url": "https://copr.stg.fedoraproject.org",
            "memory_reqs": null,
            "enable_net": true,
            "project_owner": "@copr",
            "project_name": "TEST1776291979932997799-storage-backend",
            "project_dirname": "TEST1776291979932997799-storage-backend",
            "submitter": "frostyx",
            "ended_on": 1776293429.9474413,
            "started_on": 1776292865.46147,
            "submitted_on": null,
            "status": 0,
            "chroot": "srpm-builds",
            "arch": null,
            "buildroot_pkgs": null,
            "task_id": "2925223",
            "build_id": 2925223,
            "package_name": null,
            "package_version": null,
            "git_repo": null,
            "git_hash": null,
            "git_branch": null,
            "source_type": 1,
            "source_json": "{\"url\": \"https://github.com/fedora-copr/copr-test-sources/raw/main/hello-2.8-1.src.rpm\"}",
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": null,
            "sandbox": "@copr/TEST1776291979932997799-storage-backend--frostyx",
            "results": null,
            "appstream": false,
            "allow_user_ssh": null,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [],
            "background": false,
            "distributions_in_build": [],
            "distributions_in_project": [
                "fedora-43"
            ],
            "destdir": "/var/lib/copr/public_html/results/@copr/TEST1776291979932997799-storage-backend",
            "results_repo_url": "https://download.copr-dev.fedorainfracloud.org/results/@copr/TEST1776291979932997799-storage-backend",
            "result_dir": "02925223",
            "built_packages": "",
            "tags": [
                "arch_noarch"
            ],
            "id": 2925223,
            "mockchain_macros": {
                "copr_username": "@copr",
                "copr_projectname": "TEST1776291979932997799-storage-backend",
                "vendor": "Fedora Project COPR (@copr/TEST1776291979932997799-storage-backend)"
            }
        }
    ]
}
[2026-04-15 22:50:30,049][  INFO][PID:2784953] Sending fedora-messaging bus message in build.end
[2026-04-15 22:50:31,300][  INFO][PID:2784953] Compressing /var/lib/copr/public_html/results/@copr/TEST1776291979932997799-storage-backend/srpm-builds/02925223/builder-live.log by gzip
[2026-04-15 22:50:31,306][  INFO][PID:2784953] Running command 'gzip /var/lib/copr/public_html/results/@copr/TEST1776291979932997799-storage-backend/srpm-builds/02925223/builder-live.log' as PID 2806561
[2026-04-15 22:50:31,308][  INFO][PID:2784953] Finished after 0 seconds with exit code 1 (gzip /var/lib/copr/public_html/results/@copr/TEST1776291979932997799-storage-backend/srpm-builds/02925223/builder-live.log)
stdout:

stderr:
gzip: /var/lib/copr/public_html/results/@copr/TEST1776291979932997799-storage-backend/srpm-builds/02925223/builder-live.log: No such file or directory


[2026-04-15 22:50:31,311][ ERROR][PID:2784953] Unable to compress file /var/lib/copr/public_html/results/@copr/TEST1776291979932997799-storage-backend/srpm-builds/02925223/builder-live.log
[2026-04-15 22:50:31,313][  INFO][PID:2784953] Compressing /var/lib/copr/public_html/results/@copr/TEST1776291979932997799-storage-backend/srpm-builds/02925223/backend.log by gzip
[2026-04-15 22:50:31,316][  INFO][PID:2784953] Running command 'gzip /var/lib/copr/public_html/results/@copr/TEST1776291979932997799-storage-backend/srpm-builds/02925223/backend.log' as PID 2806562