[2026-04-15 23:05:30,438][ INFO][PID:2844036] Marking build as starting [2026-04-15 23:05:30,517][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:05:30,525][ INFO][PID:2844036] VM allocation process starts [2026-04-15 23:05:30,566][ INFO][PID:2844036] Trying to allocate VM: ResallocHost, ticket_id=3673, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-15 23:05:35,571][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:05:40,575][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:05:45,580][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:05:50,584][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:05:55,587][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:06:00,596][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:06:05,607][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:06:10,613][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:06:15,619][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:06:20,626][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:06:25,632][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:06:30,639][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:06:35,648][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:06:40,650][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:06:45,653][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:06:50,655][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:06:55,658][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:07:00,659][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:07:05,664][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:07:10,671][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:07:15,676][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:07:20,681][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:07:25,689][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:07:30,696][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:07:35,704][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:07:40,712][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:07:45,719][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:07:48,883][ INFO][PID:2844036] Allocated host ResallocHost, ticket_id=3673, hostname=13.222.52.29, name=aws_aarch64_spot_dev_02147582_20260415_230510, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-15 23:07:48,884][ INFO][PID:2844036] Allocating ssh connection to builder [2026-04-15 23:07:48,885][ INFO][PID:2844036] Checking that builder machine is OK [2026-04-15 23:07:49,274][ INFO][PID:2844036] Installed copr-rpmbuild version: 1.6 [2026-04-15 23:07:49,275][ ERROR][PID:2844036] Re-try request for task on 'ResallocHost, ticket_id=3673, hostname=13.222.52.29, name=aws_aarch64_spot_dev_02147582_20260415_230510, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1 [2026-04-15 23:07:49,277][ INFO][PID:2844036] VM Release request [2026-04-15 23:07:49,293][ INFO][PID:2844036] Retry #1 (on other host) [2026-04-15 23:07:49,299][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:07:49,302][ INFO][PID:2844036] VM allocation process starts [2026-04-15 23:07:49,341][ INFO][PID:2844036] Trying to allocate VM: ResallocHost, ticket_id=3748, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-15 23:07:52,385][ INFO][PID:2844036] Allocated host ResallocHost, ticket_id=3748, hostname=13.222.52.29, name=aws_aarch64_spot_dev_02147582_20260415_230510, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-15 23:07:52,386][ INFO][PID:2844036] Allocating ssh connection to builder [2026-04-15 23:07:52,387][ INFO][PID:2844036] Checking that builder machine is OK [2026-04-15 23:07:52,433][ INFO][PID:2844036] Installed copr-rpmbuild version: 1.6 [2026-04-15 23:07:52,434][ ERROR][PID:2844036] Re-try request for task on 'ResallocHost, ticket_id=3748, hostname=13.222.52.29, name=aws_aarch64_spot_dev_02147582_20260415_230510, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1 [2026-04-15 23:07:52,435][ INFO][PID:2844036] VM Release request [2026-04-15 23:07:52,446][ INFO][PID:2844036] Retry #2 (on other host) [2026-04-15 23:07:52,446][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:07:52,447][ INFO][PID:2844036] VM allocation process starts [2026-04-15 23:07:52,485][ INFO][PID:2844036] Trying to allocate VM: ResallocHost, ticket_id=3750, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-15 23:07:57,487][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:08:02,492][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:08:07,498][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:08:12,501][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:08:17,504][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:08:22,515][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:08:27,523][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:08:32,528][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:08:37,533][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:08:42,544][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:08:47,547][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:08:52,554][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:08:57,559][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:09:02,563][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:09:07,566][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:09:12,574][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:09:17,576][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:09:22,580][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:09:27,586][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:09:32,597][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:09:37,599][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:09:40,724][ INFO][PID:2844036] Allocated host ResallocHost, ticket_id=3750, hostname=2620:52:6:1161:dead:beef:cafe:c902, name=vmhost_p09_03_dev_02147584_20260415_230602, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-15 23:09:40,724][ INFO][PID:2844036] Allocating ssh connection to builder [2026-04-15 23:09:40,726][ INFO][PID:2844036] Checking that builder machine is OK [2026-04-15 23:09:41,711][ INFO][PID:2844036] Installed copr-rpmbuild version: 1.6 [2026-04-15 23:09:41,712][ ERROR][PID:2844036] Re-try request for task on 'ResallocHost, ticket_id=3750, hostname=2620:52:6:1161:dead:beef:cafe:c902, name=vmhost_p09_03_dev_02147584_20260415_230602, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1 [2026-04-15 23:09:41,713][ INFO][PID:2844036] VM Release request [2026-04-15 23:09:41,724][ INFO][PID:2844036] Retry #3 (on other host) [2026-04-15 23:09:41,725][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:09:41,727][ INFO][PID:2844036] VM allocation process starts [2026-04-15 23:09:41,754][ INFO][PID:2844036] Trying to allocate VM: ResallocHost, ticket_id=3822, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-15 23:09:46,758][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:09:51,762][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:09:56,766][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:10:01,771][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:10:06,773][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:10:11,784][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:10:16,787][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:10:21,793][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:10:26,795][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:10:31,801][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:10:36,806][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:10:41,808][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:10:46,810][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:10:51,818][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:10:56,823][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:11:01,827][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:11:06,832][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:11:11,835][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:11:16,841][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:11:21,847][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:11:26,850][ INFO][PID:2844036] Checking for cancel request [2026-04-15 23:11:30,010][ INFO][PID:2844036] Allocated host ResallocHost, ticket_id=3822, hostname=2620:52:6:1161:dead:beef:cafe:c906, name=vmhost_x86_04_dev_02147595_20260415_230858, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-15 23:11:30,012][ INFO][PID:2844036] Allocating ssh connection to builder [2026-04-15 23:11:30,015][ INFO][PID:2844036] Checking that builder machine is OK [2026-04-15 23:11:30,616][ INFO][PID:2844036] Installed copr-rpmbuild version: 1.6 [2026-04-15 23:11:30,617][ ERROR][PID:2844036] Re-try request for task on 'ResallocHost, ticket_id=3822, hostname=2620:52:6:1161:dead:beef:cafe:c906, name=vmhost_x86_04_dev_02147595_20260415_230858, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1 [2026-04-15 23:11:30,620][ INFO][PID:2844036] VM Release request [2026-04-15 23:11:30,636][ ERROR][PID:2844036] Backend process error: Three host tried without success: {'13.222.52.29', '2620:52:6:1161:dead:beef:cafe:c906', '2620:52:6:1161:dead:beef:cafe:c902'} [2026-04-15 23:11:30,639][WARNING][PID:2844036] Switching not-finished job state to 'failed' [2026-04-15 23:11:30,641][ INFO][PID:2844036] Worker failed build, took 360.2128164768219 [2026-04-15 23:11:30,644][ INFO][PID:2844036] Sending build state back to frontend: { "builds": [ { "timeout": 108000, "frontend_base_url": "https://copr.stg.fedoraproject.org", "memory_reqs": null, "enable_net": true, "project_owner": "@copr", "project_name": "TEST1776292405237201652BuildSpecNetworking", "project_dirname": "TEST1776292405237201652BuildSpecNetworking", "submitter": "frostyx", "ended_on": 1776294690.6392055, "started_on": 1776294330.426389, "submitted_on": null, "status": 0, "chroot": "srpm-builds", "arch": null, "buildroot_pkgs": null, "task_id": "2925251", "build_id": 2925251, "package_name": null, "package_version": null, "git_repo": null, "git_hash": null, "git_branch": null, "source_type": 2, "source_json": "{\"url\": \"https://copr.stg.fedoraproject.org/tmp/tmpq89b8f2h/enum.spec\", \"pkg\": \"enum.spec\", \"tmp\": \"tmpq89b8f2h\"}", "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": null, "sandbox": "@copr/TEST1776292405237201652BuildSpecNetworking--frostyx", "results": null, "appstream": false, "allow_user_ssh": null, "ssh_public_keys": null, "storage": null, "repos": [], "background": false, "distributions_in_build": [], "distributions_in_project": [ "fedora-43" ], "destdir": "/var/lib/copr/public_html/results/@copr/TEST1776292405237201652BuildSpecNetworking", "results_repo_url": "https://download.copr-dev.fedorainfracloud.org/results/@copr/TEST1776292405237201652BuildSpecNetworking", "result_dir": "02925251", "built_packages": "", "tags": [ "arch_noarch" ], "id": 2925251, "mockchain_macros": { "copr_username": "@copr", "copr_projectname": "TEST1776292405237201652BuildSpecNetworking", "vendor": "Fedora Project COPR (@copr/TEST1776292405237201652BuildSpecNetworking)" } } ] } [2026-04-15 23:11:30,725][ INFO][PID:2844036] Sending fedora-messaging bus message in build.end [2026-04-15 23:11:32,774][ INFO][PID:2844036] Compressing /var/lib/copr/public_html/results/@copr/TEST1776292405237201652BuildSpecNetworking/srpm-builds/02925251/builder-live.log by gzip [2026-04-15 23:11:32,781][ INFO][PID:2844036] Running command 'gzip /var/lib/copr/public_html/results/@copr/TEST1776292405237201652BuildSpecNetworking/srpm-builds/02925251/builder-live.log' as PID 2860469 [2026-04-15 23:11:32,787][ INFO][PID:2844036] Finished after 0 seconds with exit code 1 (gzip /var/lib/copr/public_html/results/@copr/TEST1776292405237201652BuildSpecNetworking/srpm-builds/02925251/builder-live.log) stdout: stderr: gzip: /var/lib/copr/public_html/results/@copr/TEST1776292405237201652BuildSpecNetworking/srpm-builds/02925251/builder-live.log: No such file or directory [2026-04-15 23:11:32,788][ ERROR][PID:2844036] Unable to compress file /var/lib/copr/public_html/results/@copr/TEST1776292405237201652BuildSpecNetworking/srpm-builds/02925251/builder-live.log [2026-04-15 23:11:32,791][ INFO][PID:2844036] Compressing /var/lib/copr/public_html/results/@copr/TEST1776292405237201652BuildSpecNetworking/srpm-builds/02925251/backend.log by gzip [2026-04-15 23:11:32,799][ INFO][PID:2844036] Running command 'gzip /var/lib/copr/public_html/results/@copr/TEST1776292405237201652BuildSpecNetworking/srpm-builds/02925251/backend.log' as PID 2860472