Import/Export Orphaned uploads causing verification failures
Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.
Summary
BulkImports::ExportUpload records are being bulk deleted without removing corresponding Upload records, causing verification failures with "undefined method 'underscore' for NilClass:Class" error
Steps to reproduce
- Set-up Geo
- Import multiple projects (e.g., from Bitbucket)
- After successful import
- Check the verification status for uploads on the Geo Secondary site. You'll note all import/export related files that were supposed to be deleted now appear as failed upload verification.
What is the current bug behavior?
- Upload verification fails with
undefined method 'underscore' for NilClass:Classerror - The error occurs when attempting to verify uploads whose associated model record has been deleted.
What is the expected correct behavior?
- Upload records should be automatically removed when their associated model record is deleted
- No orphaned upload records should remain in the system
Relevant logs and/or screenshots
Results of GitLab environment info
Expand for output related to GitLab environment info
``` gitlab-ee 17.5.1 Component Installed Version Version GUID -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- alertmanager v0.27.0 git:0aa3c2aad14cff039931923ab16b26b7481783b5 bzip2 1.0.8 sha512:083f5e675d73f3233c7930ebe20425a533feedeaaa9d8cc86831312a6581cefbe6ed0d08d2fa89be81082f2a5abdabca8b3c080bf97218a1bd59dc118a30b9f3 cacerts 2022.07.19 sha256:6ed95025fba2aef0ce7b647607225745624497f876d74ef6ec22b26e73e9de77 chef-acme v4.1.6 git:c670bfbcdd7957fa052872c23e38dfba7eef02dc chef-gem 18.3.0 compat_resource v12.19.1 git:e36200f6b804915b68a4ce74c8b7a293c041d9fe config_guess c9092d05347c925a26f6887980e185206e13f9d6 git:00b15927496058d23e6258a28d8996f87cf1f191 consul v1.18.2 git:9fc827caf808b6ef1dc8c0748c3c4dce95d5b33a curl curl-8_10_1 git:7eb8c048470ed2cc14dca75be9c1cdae7ac8498b docker-distribution-pruner v0.2.0 git:a796e3670d508529da84ac66f242d87ff7803609 exiftool 12.96 git:0f63a780906abcccba796761fc2e66a0737e2f16 git v17.5.1 git:42b2dc21d50d19003ae74ba9cdadc94aed686162 git-filter-repo v2.45.0 gitaly v17.5.1 git:42b2dc21d50d19003ae74ba9cdadc94aed686162 gitlab-backup-cli 11bd2a07e00a1c1de5d6a7dfe4301fe4 gitlab-config-template 17.5.1 gitlab-cookbooks 17.5.1 gitlab-ctl 17.5.1 gitlab-ctl-ee 17.5.1 gitlab-elasticsearch-indexer v5.3.0 git:a4aab7fbaa46dc99abe611a2a460c32bedd7fa36 gitlab-exporter 15.0.0 gitlab-geo-psql 829ff725af11406efef31cfe6255afcb gitlab-healthcheck 0060154a010637bec197fb38217ddcbc gitlab-kas v17.5.1 git:a4846fa38b47638ef3d37ecc9ee0ebaaad8e88db gitlab-pages v17.5.1 git:ef355aa9cb110e9d568ee62432ab324330b7399c gitlab-pg-ctl 708eef37ad59a61728d790c485aae4a6 gitlab-psql 4ac3cc368e73f3750591493104798fce gitlab-rails v17.5.1-ee git:e4e13234b2da1da5a93791b06b425f36809bf4a6 gitlab-redis-cli e16c5ba796580cf0c774589f65392f65 gitlab-scripts 17.5.1 gitlab-shell v14.39.0 git:2ff6909d8c43a78f34b698a863e554a7bc301545 gnupg 2.2.41 sha256:13f3291007a5e8546fcb7bc0c6610ce44aaa9b3995059d4f8145ba09fd5be3e1 go-crond 23.12.0 git:4f2e74e2a5544a72c6523c22f687d72be593e981 gpgme 1.18.0 sha256:361d4eae47ce925dba0ea569af40e7b52c645c4ae2e65e5621bf1b6cdd8b0e9e graphicsmagick 1.3.36 sha256:1e6723c48c4abbb31197fadf8396b2d579d97e197123edc70a4f057f0533d563 inspec-gem 6.6.0 jemalloc 5.3.0 git:2a693b83d2d1631b6a856d178125e1c47c12add9 krb5 krb5-1.17 git:b399721b5aedacc490158c04f6a3fa77c98f0b62 libarchive 3.7.5 sha256:37556113fe44d77a7988f1ef88bf86ab68f53d11e85066ffd3c70157cc5110f1 libassuan 2.5.5 sha256:8e8c2fcc982f9ca67dcbb1d95e2dc746b1739a4668bc20b3a3c5be632edb34e4 libedit 20120601-3.0 sha256:51f0f4b4a97b7ebab26e7b5c2564c47628cdb3042fd8ba8d0605c719d2541918 libevent release-2.1.12-stable git:5df3037d10556bfcb675bc73e516978b75fc7bc7 libffi 3.2.1 sha256:d06ebb8e1d9a22d19e38d63fdb83954253f39bedc5d46232a05645685722ca37 libgcrypt 1.9.4 sha256:ea849c83a72454e3ed4267697e8ca03390aee972ab421e7df69dfe42b65caaf7 libgpg-error 1.46 sha256:b7e11a64246bbe5ef37748de43b245abd72cfcd53c9ae5e7fc5ca59f1c81268d libiconv 1.15 sha256:ccf536620a45458d26ba83887a983b96827001e92a13847b45e4925cc8913178 libicu release-63-1 git:46895456ad1b6660d17eaeba2c101600ad8d8eb8 libjpeg-turbo 2.1.5.1 git:8ecba3647edb6dd940463fedf38ca33a8e2a73d1 libksba 1.6.3 sha256:3f72c68db30971ebbf14367527719423f0a4d5f8103fc9f4a1c01a9fa440de5c liblzma 5.2.4 sha256:b512f3b726d3b37b6dc4c8570e137b9311e7552e8ccbab4d39d47ce5f4177145 libossp-uuid 1.6.2 sha256:11a615225baa5f8bb686824423f50e4427acd3f70d394765bdff32801f0fd5b0 libpng v1.6.44 git:f5e92d76973a7a53f517579bc95d61483bf108c0 libtensorflow_lite v2.6.0 git:16692e241ee57fbbb447810b124f7ba406d65eba libtiff v4.7.0 git:9dff73bebc5661f2dace6f16e14cf9e857172f4e libtool 2.4.6 sha256:e3bd4d5d3d025a36c21dd6af7ea818a2afcd4dfc1ea5a17b39d7854bcd0c06e3 libxml2 2.13.4 sha256:65d042e1c8010243e617efb02afda20b85c2160acdbfbcb5b26b80cec6515650 libxslt 1.1.41 sha256:3ad392af91115b7740f7b50d228cc1c5fc13afc1da7f16cb0213917a37f71bda libyaml 0.2.5 sha256:c642ae9b75fee120b2d96c712538bd2cf283228d2337df2cf2988e3c02678ef4 logrotate 3.21.0 git:5429847b018ba9d985eab85e837fbfd37e695d3a mattermost 10.0.1 sha256:932090fd6fbc44e33ddcdc57f89fe1048c3a1d486de3b0395839aef47a182b77 ncurses 6315e1a380ecdb706d4f6518d2e8c7eb0db8fbe2 git:87c2c84cbd2332d6d94b12a1dcaf12ad1a51a938 nginx release-1.24.0 git:84cd7217778b8b3a2e395194b641ea14050344f1 nginx-module-vts v0.2.2 git:724b34d7f1eff083860e3ac613a2c1d66a238dfd ngx_security_headers 0.0.9 git:390dc7cd0e66118412d3c7a5141422dd5795511a node-exporter v1.8.2 git:f1e0e8360aa60b6cb5e5cc1560bed348fc2c1895 npth 1.6 sha256:1393abd9adcf0762d34798dc34fdcf4d0d22a8410721e76f1e3afcd1daa4e2d1 omnibus-ctl v0.6.12 git:4e968f4cf713a545bbb70ee13b8ff7ebd37f7f3e omnibus-gitlab-gems 20240619 openssl OpenSSL_1_1_1w git:f1607c8a2c04bcb95ddb2e6fc4e0aaec9729929b package-scripts 17.5.1+ee.0 patroni 3.0.1 pcre 8.44 sha256:aecafd4af3bd0f3935721af77b889d9024b2e01d96b58471bd91a3063fb47728 pcre2 pcre2-10.44 git:6ae58beca071f13ccfed31d03b3f479ab520639b pgbouncer pgbouncer_1_22_1 git:de1e9cb75324a624d18fc3dd5d714d12a8da0db6 pgbouncer-exporter v0.9.0 git:5716a88b8dc8ce887878b2bda832ff46d79f67d2 pkg-config-lite 0.28-1 sha256:21b76ec4e115ee30f9b3077a2506e48e8b837332ed4d30c9776502e69c6a29e5 popt 1.16 sha256:e728ed296fe9f069a0e005003c3d6b2dde3d9cad453422a10d6558616d304cc8 postgres-exporter v0.15.0 git:68c176b8833b7580bf847cecf60f8e0ad5923f9a postgresql 14.11 sha256:a670bd7dce22dcad4297b261136b3b1d4a09a6f541719562aa14ca63bf2968a8 postgresql_new 16.4 sha256:971766d645aa73e93b9ef4e3be44201b4f45b5477095b049125403f9f3386d6f prometheus v2.54.1 git:e6cfa720fbe6280153fab13090a483dbd40bece3 psycopg2 2_8_6 git:b203be11a69789a50da828a0a5e4c15d2dcf99d4 python-docutils 0.19 python3 3.9.17 sha256:8ead58f669f7e19d777c3556b62fae29a81d7f06a7122ff9bc57f7dd82d7e014 rb-readline master git:9fba246073f78831b7c7129c76cc07d8476a8892 redis 7.0.15 git:05827336b0efcb454bcc5fe77ceacc6f226b6bb2 redis-exporter v1.63.0 git:3526bf717ef97617132b64701cd112f21be13cd0 registry v4.10.0-gitlab git:09a142a8a223c6d701c7c2667f83cba00ef04427 rsync 3.2.7 sha256:4e7d9d3f6ed10878c58c5fb724a67dacf4b6aac7340b13e488fb2dc41346f2bb ruby 3.2.5 sha256:ef0610b498f60fb5cfd77b51adb3c10f4ca8ed9a17cb87c61e5bea314ac34a16 rubygems 3.5.17 runit 2.1.2 sha256:6fd0160cb0cf1207de4e66754b6d39750cff14bb0aa66ab49490992c0c47ba18 spam-classifier 0.3.0 sha256:c9f7e147d195a45e32c35765e138e006e7636218f8c4413e67d0cef5513335a8 spamcheck v0.3.2 git:e1d8002ff4fbff2f5b0c8baa6d381607a2863f7f unzip 6.0.27 sha256:036d96991646d0449ed0aa952e4fbe21b476ce994abc276e49d30e686708bd37 version-manifest 0.0.1 zlib v1.3.1 git:51b7f2abdade71cd9bb0e7a373ef2610ec6f9daf ```
Results of GitLab application Check
Expand for output related to the GitLab application check
\`\`\`\` gitlab-rake gitlab:uploads:check Checking integrity of Uploads - 4..205: Failures: 0 - 206..405: Failures: 0 - 406..605: Failures: 0 - 606..805: Failures: 0 - 806..1005: Failures: 0 - 1006..1205: Failures: 0 - 1206..1405: Failures: 0 - 1406..1605: Failures: 0 - 1606..1805: Failures: 0 - 1806..2005: Failures: 0 - 2006..2205: Failures: 0 - 2206..2405: Failures: 0 - 2406..2612: Failures: 0 - 2613..2643: Failures: 0 Done!\`\`\` \`\`\`\`
Possible fixes
Edited by 🤖 GitLab Bot 🤖