-
-
Notifications
You must be signed in to change notification settings - Fork 5.8k
Horrible Dashboard Delay After Upgrade #32224
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
I will investigate it. How many records in the table action. |
Hey @lunny, I have a somewhat large instance of Gitea running on my machine (at least I think it is large -- its actions table has almost 13 million rows). I've created an index based on the action's user_id and is_deleted columns and the cost of the queries has dropped from 61923.66 to about 8). The database is Postgres. I don't know if such an index would have other implications that I'm unaware of. But if you think that adding the index would be a sensible approach, I would gladly submit a PR for it. |
According to the current index function, I think maybe you are right, we need that index. Please send a PR to fix this. // TableIndices implements xorm's TableIndices interface
func (a *Action) TableIndices() []*schemas.Index {
repoIndex := schemas.NewIndex("r_u_d", schemas.IndexType)
repoIndex.AddColumn("repo_id", "user_id", "is_deleted")
actUserIndex := schemas.NewIndex("au_r_c_u_d", schemas.IndexType)
actUserIndex.AddColumn("act_user_id", "repo_id", "created_unix", "user_id", "is_deleted")
cudIndex := schemas.NewIndex("c_u_d", schemas.IndexType)
cudIndex.AddColumn("created_unix", "user_id", "is_deleted")
indices := []*schemas.Index{actUserIndex, repoIndex, cudIndex}
return indices
} |
I think the |
Looks like we already have the index definition. |
Nope, it is more like an arbitrary unit of costs. This article does a great job of explaining the concept.
For sure. The first screen shot shows the cost when using the indexes that are already in place. See that it uses the For fetching whe get the results below. If I apply the command The results below show the fetching results. As of what is happening with the Fetching results: If the |
Can you try |
In my case I see improvements which was made in #31752. In my action table are 3977694 entries. And this Idea: The count query could by cached and invalidated when there are changes in the action table? this would speedup this. Depends how often something changes in the action table. I missing only some improvements on the higher pages on pagging. Cause there sorting is consuming too much RAM. I mean this query
And the optimizes query need less RAM |
How is your Postgres RAM configuration? In my case I saw a lot of tmp-files and I increased some parameters.
This was an improvement. And this https://explain.depesz.com/ make the explain result more readable. |
For sure. I just added limit to the end of the query (since the dashboard is paged, I assume it is) The original index yields the results below: In the other hand, we get the following results when using the (user_id, is_deleted) index. Notice that postgre still uses the same index. After that, I've added created the (user_id, is_deleted, created_unix) index. Which resulted in: Lastly, I've tried to create the index as (is_deleted, user_id, created_unix) index. The results was: Tha last two index are pratically equal in terms of performance (the actual cost of the 4th being marginally lower).
I'll have to check it out. But I'm pretty sure no tuning was made to this particular database since it used mainly for development. BTW, nice tool :) |
The index sequence maybe affect the final performance result. I guess index |
Yeap. I think the main culprit is indeed the order of the columns in the index. If you guys are okay with it. I think I can supply a PR for that during the weekend. |
I added this
But I can't see, that this will be used. SELECT count(*) FROM "action" WHERE user_id=1 AND is_deleted='f' : and SELECT |
There are 847525 records in the action table. |
@lunny is this still a thing ? It flew completely under my radar :( |
Yes, I think maybe we need a new index for |
I created #32333 to resolve the problem. |
@lunny I have my 2nd test instance with ~19.000 mirrors. I need then an linux-amd64 executable. |
and I thought I was silly being with 6900 mirrors. using the docker image... If there's a nighty build or docker image I'm happy to try it |
https://gitea.com/lunny/packages/src/branch/main/gitea-pr-32333-amd64-linux-1.tar.gz |
@lunny thx. I try this later today. |
I started it
The new pr version feels fast. vs and vs and on higher pages vs Speedup ist good. Sorting is the same. Works for me! |
My mistake was: I didn't install 1.22.3 for comparison. |
@lunny I just mounted your binary into my docker container |
Yes, my binary doesn't support SQLite3 because I compile it in macOS. |
Here 1.22.3 results ... "same" performance like with 1.22.2. The performance in the https://gitea.com/lunny/packages/src/branch/main/gitea-pr-32333-amd64-linux-1.tar.gz versions is better. |
Manually adding that index seems to have resolved my performance problems, went from around 13+ second loads to less than 1 second on the dashboard. |
Something of an improvement. It took a couple of minutes to add the index, and a couple of minutes (123000ms) to log in, but after that index page loads in 10s instead of 90s. Subsequent logins take 10s instead of 120s, so yeah... about 12x faster, like @SoulSeekkor sees
|
Description
Previously on 1.22.2 the dashboard for my instance loaded instantly, after upgrading to 1.22.3 when I load/refresh the dashboard the refreshing simply hangs (web page does nothing) for over 8 seconds then the entire dashboard is shown at once. Running Windows/Ubuntu with a MSSQL backend doesn't seem to encounter this problem, only with the MySQL8 backend is it doing this.
Relevant logs:
2024/10/09 08:45:53 cmd/web.go:194:serveInstalled() [W] Table oauth2_application column redirect_uris db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:53 cmd/web.go:194:serveInstalled() [W] Table oauth2_grant column scope db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:53 cmd/web.go:194:serveInstalled() [W] Table oauth2_grant column nonce db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:53 cmd/web.go:194:serveInstalled() [W] Table login_source column cfg db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:53 cmd/web.go:194:serveInstalled() [W] Table notice column description db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:53 cmd/web.go:194:serveInstalled() [W] Table system_setting Column version db default is , struct default is 1
2024/10/09 08:45:53 cmd/web.go:194:serveInstalled() [W] Table email_address Column lower_email db nullable is true, struct nullable is false
2024/10/09 08:45:53 cmd/web.go:194:serveInstalled() [W] Table external_login_user column raw_data db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:53 cmd/web.go:194:serveInstalled() [W] Table external_login_user column avatar_url db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:53 cmd/web.go:194:serveInstalled() [W] Table external_login_user column access_token db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:53 cmd/web.go:194:serveInstalled() [W] Table external_login_user column access_token_secret db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:53 cmd/web.go:194:serveInstalled() [W] Table external_login_user column refresh_token db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:53 cmd/web.go:194:serveInstalled() [W] Table user Column keep_activity_private db default is , struct default is false
2024/10/09 08:45:53 cmd/web.go:194:serveInstalled() [W] Table user Column keep_activity_private db nullable is true, struct nullable is false
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table language_stat Column language db nullable is true, struct nullable is false
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table mirror column lfs_endpoint db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table release column note db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table repository column description db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table repository Column original_service_type db default is 0, struct default is
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table repository column topics db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table repo_unit column config db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table task column payload_content db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table task column message db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table gpg_key column content db type is LONGTEXT(4294967295), struct type is MEDIUMTEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table gpg_key Column content db nullable is true, struct nullable is false
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table gpg_key column emails db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table gpg_key_import column content db type is LONGTEXT(4294967295), struct type is MEDIUMTEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table gpg_key_import Column content db nullable is true, struct nullable is false
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table public_key column content db type is LONGTEXT(4294967295), struct type is MEDIUMTEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table public_key Column content db nullable is true, struct nullable is false
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table project column board_type db type is INT, struct type is INT UNSIGNED
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table project column card_type db type is INT, struct type is INT UNSIGNED
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table project Column card_type db default is 0, struct default is
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table project Column card_type db nullable is false, struct nullable is true
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table project column type db type is INT, struct type is INT UNSIGNED
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table review_state Column commit_sha db nullable is true, struct nullable is false
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table hook_task has column repo_id but struct has not related field
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table webhook Column is_system_webhook db default is 0, struct default is
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table webhook Column is_system_webhook db nullable is false, struct nullable is true
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table webhook column url db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table webhook Column http_method db default is 'POST', struct default is
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table webhook column secret db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table webhook column events db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table webhook column meta db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table webhook column header_authorization_encrypted db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table commit_status column target_url db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table commit_status column description db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table commit_status column context db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table lfs_lock column path db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table protected_branch column whitelist_user_i_ds db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table protected_branch column whitelist_team_i_ds db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table protected_branch column merge_whitelist_user_i_ds db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table protected_branch column merge_whitelist_team_i_ds db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table protected_branch column status_check_contexts db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table protected_branch column approvals_whitelist_user_i_ds db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table protected_branch column approvals_whitelist_team_i_ds db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table protected_branch column protected_file_patterns db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table protected_branch column unprotected_file_patterns db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table comment has column old_assignee_id but struct has not related field
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table label Column archived_unix db default is , struct default is NULL
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table milestone column content db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table pull_request column conflicted_files db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table pull_request column changed_protected_files db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table review column type db type is VARCHAR(255), struct type is INT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table review column content db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table tracked_time Column time db nullable is true, struct nullable is false
2024/10/09 08:45:54 cmd/web.go:194:serveInstalled() [W] Table action column content db type is MEDIUMTEXT(16777215), struct type is TEXT
2024/10/09 08:46:24 ...eb/routing/logger.go:68:func1() [W] router: slow GET / for 10.12.10.6:53490, elapsed 3197.1ms @ web/home.go:32(web.Home)
2024/10/09 08:46:29 ...activities/action.go:459:GetFeeds() [W] [Slow SQL Query] SELECT count(*) FROM
action
WHERE user_id=? AND is_deleted=? [1 false] - 8.310601896s2024/10/09 08:46:29 ...ls/avatars/avatar.go:120:LibravatarURL() [E] LibravatarService.FromEmail(email=github-actions[bot]@users.noreply.github.com): error mail: missing '@' or angle-addr
2024/10/09 08:46:32 ...eful/manager_unix.go:144:handleSignals() [W] PID 448390. Received SIGTERM. Shutting down...
2024/10/09 08:46:32 .../graceful/manager.go:168:doHammerTime() [W] Setting Hammer condition
2024/10/09 08:46:33 .../graceful/manager.go:184:doTerminate() [W] Terminating
2024/10/09 08:46:33 ...eful/manager_unix.go:154:handleSignals() [W] PID: 448390. Background context for manager closed - context canceled - Shutting down...
Gitea Version
1.22.3
Can you reproduce the bug on the Gitea demo site?
No
Log Gist
No response
Screenshots
No response
Git Version
2.46.2
Operating System
Ubuntu 24.04
How are you running Gitea?
This is with the published release binary for 64-bit Linux running in a Ubuntu 24.04 incus (LXC) container.
Database
MySQL 8
The text was updated successfully, but these errors were encountered: