[2026-04-15 23:14:10,009][ INFO][PID:2867188] Marking build as starting [2026-04-15 23:14:10,133][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:14:10,142][ INFO][PID:2867188] VM allocation process starts [2026-04-15 23:14:10,172][ INFO][PID:2867188] Trying to allocate VM: ResallocHost, ticket_id=3986, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-15 23:14:15,177][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:14:20,178][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:14:25,181][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:14:30,184][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:14:35,189][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:14:40,193][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:14:45,199][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:14:50,202][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:14:55,208][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:15:00,210][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:15:05,213][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:15:10,214][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:15:15,216][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:15:20,221][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:15:25,223][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:15:30,231][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:15:35,241][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:15:38,431][ INFO][PID:2867188] Allocated host ResallocHost, ticket_id=3986, hostname=100.30.171.182, name=aws_aarch64_spot_dev_02147613_20260415_231339, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-15 23:15:38,432][ INFO][PID:2867188] Allocating ssh connection to builder [2026-04-15 23:15:38,437][ INFO][PID:2867188] Checking that builder machine is OK [2026-04-15 23:15:38,922][ INFO][PID:2867188] Installed copr-rpmbuild version: 1.6 [2026-04-15 23:15:38,925][ ERROR][PID:2867188] Re-try request for task on 'ResallocHost, ticket_id=3986, hostname=100.30.171.182, name=aws_aarch64_spot_dev_02147613_20260415_231339, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1 [2026-04-15 23:15:38,926][ INFO][PID:2867188] VM Release request [2026-04-15 23:15:38,956][ INFO][PID:2867188] Retry #1 (on other host) [2026-04-15 23:15:38,958][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:15:38,964][ INFO][PID:2867188] VM allocation process starts [2026-04-15 23:15:39,010][ INFO][PID:2867188] Trying to allocate VM: ResallocHost, ticket_id=4027, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-15 23:15:44,022][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:15:49,027][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:15:54,034][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:15:59,036][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:16:04,044][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:16:09,050][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:16:14,057][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:16:19,060][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:16:24,065][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:16:29,069][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:16:34,072][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:16:39,075][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:16:44,080][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:16:49,081][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:16:54,086][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:16:59,089][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:17:04,094][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:17:09,101][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:17:14,105][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:17:19,108][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:17:24,113][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:17:27,314][ INFO][PID:2867188] Allocated host ResallocHost, ticket_id=4027, hostname=2620:52:6:1161:dead:beef:cafe:c902, name=vmhost_p09_03_dev_02147614_20260415_231340, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-15 23:17:27,315][ INFO][PID:2867188] Allocating ssh connection to builder [2026-04-15 23:17:27,316][ INFO][PID:2867188] Checking that builder machine is OK [2026-04-15 23:17:28,482][ INFO][PID:2867188] Installed copr-rpmbuild version: 1.6 [2026-04-15 23:17:28,483][ ERROR][PID:2867188] Re-try request for task on 'ResallocHost, ticket_id=4027, hostname=2620:52:6:1161:dead:beef:cafe:c902, name=vmhost_p09_03_dev_02147614_20260415_231340, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1 [2026-04-15 23:17:28,487][ INFO][PID:2867188] VM Release request [2026-04-15 23:17:28,556][ INFO][PID:2867188] Retry #2 (on other host) [2026-04-15 23:17:28,559][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:17:28,561][ INFO][PID:2867188] VM allocation process starts [2026-04-15 23:17:28,636][ INFO][PID:2867188] Trying to allocate VM: ResallocHost, ticket_id=4065, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-15 23:17:31,707][ INFO][PID:2867188] Allocated host ResallocHost, ticket_id=4065, hostname=2620:52:6:1161:dead:beef:cafe:c902, name=vmhost_p09_03_dev_02147614_20260415_231340, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-15 23:17:31,710][ INFO][PID:2867188] Allocating ssh connection to builder [2026-04-15 23:17:31,713][ INFO][PID:2867188] Checking that builder machine is OK [2026-04-15 23:17:31,861][ INFO][PID:2867188] Installed copr-rpmbuild version: 1.6 [2026-04-15 23:17:31,861][ ERROR][PID:2867188] Re-try request for task on 'ResallocHost, ticket_id=4065, hostname=2620:52:6:1161:dead:beef:cafe:c902, name=vmhost_p09_03_dev_02147614_20260415_231340, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1 [2026-04-15 23:17:31,862][ INFO][PID:2867188] VM Release request [2026-04-15 23:17:31,886][ INFO][PID:2867188] Retry #3 (on other host) [2026-04-15 23:17:31,887][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:17:31,891][ INFO][PID:2867188] VM allocation process starts [2026-04-15 23:17:31,934][ INFO][PID:2867188] Trying to allocate VM: ResallocHost, ticket_id=4067, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-15 23:17:35,028][ INFO][PID:2867188] Allocated host ResallocHost, ticket_id=4067, hostname=2620:52:6:1161:dead:beef:cafe:c902, name=vmhost_p09_03_dev_02147614_20260415_231340, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-15 23:17:35,029][ INFO][PID:2867188] Allocating ssh connection to builder [2026-04-15 23:17:35,030][ INFO][PID:2867188] Checking that builder machine is OK [2026-04-15 23:17:35,156][ INFO][PID:2867188] Installed copr-rpmbuild version: 1.6 [2026-04-15 23:17:35,158][ ERROR][PID:2867188] Re-try request for task on 'ResallocHost, ticket_id=4067, hostname=2620:52:6:1161:dead:beef:cafe:c902, name=vmhost_p09_03_dev_02147614_20260415_231340, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1 [2026-04-15 23:17:35,158][ INFO][PID:2867188] VM Release request [2026-04-15 23:17:35,174][ INFO][PID:2867188] Retry #4 (on other host) [2026-04-15 23:17:35,176][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:17:35,177][ INFO][PID:2867188] VM allocation process starts [2026-04-15 23:17:35,237][ INFO][PID:2867188] Trying to allocate VM: ResallocHost, ticket_id=4069, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-15 23:17:38,290][ INFO][PID:2867188] Allocated host ResallocHost, ticket_id=4069, hostname=2620:52:6:1161:dead:beef:cafe:c902, name=vmhost_p09_03_dev_02147614_20260415_231340, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-15 23:17:38,291][ INFO][PID:2867188] Allocating ssh connection to builder [2026-04-15 23:17:38,293][ INFO][PID:2867188] Checking that builder machine is OK [2026-04-15 23:17:38,439][ INFO][PID:2867188] Installed copr-rpmbuild version: 1.6 [2026-04-15 23:17:38,439][ ERROR][PID:2867188] Re-try request for task on 'ResallocHost, ticket_id=4069, hostname=2620:52:6:1161:dead:beef:cafe:c902, name=vmhost_p09_03_dev_02147614_20260415_231340, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1 [2026-04-15 23:17:38,442][ INFO][PID:2867188] VM Release request [2026-04-15 23:17:38,469][ INFO][PID:2867188] Retry #5 (on other host) [2026-04-15 23:17:38,472][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:17:38,473][ INFO][PID:2867188] VM allocation process starts [2026-04-15 23:17:38,495][ INFO][PID:2867188] Trying to allocate VM: ResallocHost, ticket_id=4071, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-15 23:17:43,500][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:17:48,503][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:17:53,510][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:17:58,517][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:18:03,521][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:18:08,524][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:18:13,531][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:18:18,536][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:18:23,541][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:18:28,544][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:18:33,546][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:18:38,551][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:18:43,555][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:18:48,556][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:18:53,558][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:18:58,562][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:19:03,565][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:19:08,566][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:19:13,571][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:19:18,577][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:19:23,583][ INFO][PID:2867188] Checking for cancel request [2026-04-15 23:19:26,682][ INFO][PID:2867188] Allocated host ResallocHost, ticket_id=4071, hostname=2620:52:6:1161:dead:beef:cafe:c906, name=vmhost_x86_02_dev_02147627_20260415_231720, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-15 23:19:26,683][ INFO][PID:2867188] Allocating ssh connection to builder [2026-04-15 23:19:26,683][ INFO][PID:2867188] Checking that builder machine is OK [2026-04-15 23:19:27,214][ INFO][PID:2867188] Installed copr-rpmbuild version: 1.6 [2026-04-15 23:19:27,215][ ERROR][PID:2867188] Re-try request for task on 'ResallocHost, ticket_id=4071, hostname=2620:52:6:1161:dead:beef:cafe:c906, name=vmhost_x86_02_dev_02147627_20260415_231720, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1 [2026-04-15 23:19:27,216][ INFO][PID:2867188] VM Release request [2026-04-15 23:19:27,227][ ERROR][PID:2867188] Backend process error: Three host tried without success: {'100.30.171.182', '2620:52:6:1161:dead:beef:cafe:c902', '2620:52:6:1161:dead:beef:cafe:c906'} [2026-04-15 23:19:27,228][WARNING][PID:2867188] Switching not-finished job state to 'failed' [2026-04-15 23:19:27,228][ INFO][PID:2867188] Worker failed build, took 317.2367687225342 [2026-04-15 23:19:27,229][ INFO][PID:2867188] Sending build state back to frontend: { "builds": [ { "timeout": 108000, "frontend_base_url": "https://copr.stg.fedoraproject.org", "memory_reqs": null, "enable_net": true, "project_owner": "@copr", "project_name": "TEST1776292405217201971TestEpel", "project_dirname": "TEST1776292405217201971TestEpel", "submitter": "frostyx", "ended_on": 1776295167.2281804, "started_on": 1776294849.9914117, "submitted_on": null, "status": 0, "chroot": "srpm-builds", "arch": null, "buildroot_pkgs": null, "task_id": "2925261", "build_id": 2925261, "package_name": null, "package_version": null, "git_repo": null, "git_hash": null, "git_branch": null, "source_type": 1, "source_json": "{\"url\": \"https://github.com/fedora-copr/copr-test-sources/raw/main/hello-2.8-1.src.rpm\"}", "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": null, "sandbox": "@copr/TEST1776292405217201971TestEpel--frostyx", "results": null, "appstream": false, "allow_user_ssh": null, "ssh_public_keys": null, "storage": null, "repos": [], "background": false, "distributions_in_build": [], "distributions_in_project": [ "epel-6", "epel-7", "epel-8", "epel-9" ], "destdir": "/var/lib/copr/public_html/results/@copr/TEST1776292405217201971TestEpel", "results_repo_url": "https://download.copr-dev.fedorainfracloud.org/results/@copr/TEST1776292405217201971TestEpel", "result_dir": "02925261", "built_packages": "", "tags": [ "arch_noarch" ], "id": 2925261, "mockchain_macros": { "copr_username": "@copr", "copr_projectname": "TEST1776292405217201971TestEpel", "vendor": "Fedora Project COPR (@copr/TEST1776292405217201971TestEpel)" } } ] } [2026-04-15 23:19:27,335][ INFO][PID:2867188] Sending fedora-messaging bus message in build.end [2026-04-15 23:19:28,384][ INFO][PID:2867188] Compressing /var/lib/copr/public_html/results/@copr/TEST1776292405217201971TestEpel/srpm-builds/02925261/builder-live.log by gzip [2026-04-15 23:19:28,387][ INFO][PID:2867188] Running command 'gzip /var/lib/copr/public_html/results/@copr/TEST1776292405217201971TestEpel/srpm-builds/02925261/builder-live.log' as PID 2880022 [2026-04-15 23:19:28,388][ INFO][PID:2867188] Finished after 0 seconds with exit code 1 (gzip /var/lib/copr/public_html/results/@copr/TEST1776292405217201971TestEpel/srpm-builds/02925261/builder-live.log) stdout: stderr: gzip: /var/lib/copr/public_html/results/@copr/TEST1776292405217201971TestEpel/srpm-builds/02925261/builder-live.log: No such file or directory [2026-04-15 23:19:28,389][ ERROR][PID:2867188] Unable to compress file /var/lib/copr/public_html/results/@copr/TEST1776292405217201971TestEpel/srpm-builds/02925261/builder-live.log [2026-04-15 23:19:28,390][ INFO][PID:2867188] Compressing /var/lib/copr/public_html/results/@copr/TEST1776292405217201971TestEpel/srpm-builds/02925261/backend.log by gzip [2026-04-15 23:19:28,392][ INFO][PID:2867188] Running command 'gzip /var/lib/copr/public_html/results/@copr/TEST1776292405217201971TestEpel/srpm-builds/02925261/backend.log' as PID 2880023