- Databases are one of the most important parts of Forgejo, every
interaction with Forgejo uses the database in one way or another.
Therefore, it is important to maintain the database and recognize when
Forgejo is not doing well with the database. Forgejo already has the
option to log *every* SQL query along with its execution time, but
monitoring becomes impractical for larger instances and takes up
unnecessary storage in the logs.
- Add a QoL enhancement that allows instance administrators to specify a
threshold value beyond which query execution time is logged as a warning
in the xorm logger. The default value is a conservative five seconds to
avoid this becoming a source of spam in the logs.
- The use case for this patch is that with an instance the size of Codeberg, monitoring SQL logs is not very fruitful and most of them are uninteresting. Recently, in the context of persistent deadlock issues (https://codeberg.org/forgejo/forgejo/issues/220), I have noticed that certain queries hold locks on tables like comment and issue for several seconds. This patch helps to identify which queries these are and when they happen.
- Added unit test.
(cherry picked from commit 24bbe7886fb4cb9a38c8dab8c44f4c9cbfa25481)
(cherry picked from commit 6e29145b3c1455498531593d38e6a914941a12cb)
(cherry picked from commit 63731e30712872bd2395eb3cf36d9996e5793645)
(cherry picked from commit 3ce1a097369c132654de70df707b867e47bd1c40)
(cherry picked from commit a64426907de788cc0937a7a2b16af4d2f26f7fe6)
(cherry picked from commit 4b1921569156445c58d9889602733da5934c7b95)
(cherry picked from commit e6356744359fa947c049827d60c2ea0e277e03dc)
(cherry picked from commit 9cf501f1af4cd870221cef6af489618785b71186)
(cherry picked from commit 0d6b934eba1c0e9b27b364791113aae816b6b366)
(cherry picked from commit 4b6c2738795002887844a106f2fed2ef1673eed1)
(cherry picked from commit 89b1315338b0c7a726a36a84e9844013a13560b8)
(cherry picked from commit edd8e66ce991c395bb0af7720631c3cd26caaa51)
[GITEA] Add slow SQL query warning (squash) document the setting
(cherry picked from commit ce38599c5141c7fc6bc054819f5ff1c1b45bda1f)
(cherry picked from commit 794aa67c68c8e24ac7301eb7ef767c6e2499a78d)
(cherry picked from commit a4c2c6b004c21488e90f637ca7920f49108ed75d)
(cherry picked from commit 97912752bc802db79bb26a6591aec885aea30ee4)
(cherry picked from commit 00b5327c9750215a290238516e7b6fb1e6601e14)
(cherry picked from commit 1069c860e78c11225b4d74ff3044df7786562821)
(cherry picked from commit 84241f42c83852918b57c8bd25364697037fe42f)
(cherry picked from commit e4bda0e8457d00c01b83f153ed5a4a8ea4cf85c8)
(cherry picked from commit 7357fb91bff87045b133c3a7ac9fc70eea781bc4)
(cherry picked from commit a8dd7f6da278ae112200b5efa5bf27e3961f5996)
(cherry picked from commit e636e9f4beca7273dd8622baedb2f0c01db30449)
(cherry picked from commit bf04ae86037f5cb5a81d02750aead2742b040367)
(cherry picked from commit 93b19e3568169bd1cf9b8b78c1751c3d2d65a1b6)
(cherry picked from commit 83f91363ad071675c73a1f636271cc043bf69707)
(cherry picked from commit e34a05bc7319072b70d387975342d617b8136655)
(cherry picked from commit 68569aeee9805aaa8e98c6e7fe8058095e290061)
(cherry picked from commit 88d1b53eeaa0d5ad0ed54c191236db928aadedf0)
(cherry picked from commit f63f71afad2aa8272772dba919485c5e858d2fae)
(cherry picked from commit 1f774145fc731e2d93cf8a6708dbf4c15722fa5e)
(cherry picked from commit 53e637693bcbb9fe9daca9a296c9d2c700fc915b)
(cherry picked from commit 4974cbf10a8b51d51591fa6f6281f4cd7a415c91)
(cherry picked from commit 554bca7fae7fa6f0235ab631598115875711d80e)
(cherry picked from commit 61b8cf83b988a2d8547b81cfb7aaeefaf2e17fba)
(cherry picked from commit d8bcc6f68c6dc972e8c99eb7b46a4f096f85cd61)
(cherry picked from commit ee04c340724cccada2c71db45ec510d19e92e099)
(cherry picked from commit 713153a6d63205984f2a85ee07f08269df3b0a85)
(cherry picked from commit 4e18c4f8bb35b02494d75311986fadb87328d477)
(cherry picked from commit 32be9db12df3dea9c12bf803a5226b6ae8cd7b32)
(cherry picked from commit fd50e9b9b4d7cd804e57418056126ed2df9bec6a)
(cherry picked from commit 11717b864bf4d820305511cf22ec50f61604a770)
(cherry picked from commit b70f3e0fa5db9cd90c961138c1ab748aa5c6a4d8)
(cherry picked from commit 2d802b2963d18b28a85a82e01f7a7f18ab8c3192)
(cherry picked from commit f61e2f71e2d629cc4e480f9e61c9b383675da0ef)
(cherry picked from commit 31723651c96d3f20811540aad94878815795da15)
(cherry picked from commit 38f6f703a0dfb04a9d436c8c806f1be513c7ea3a)
(cherry picked from commit 75f6716905755cbe3d7e49d965ab81a47f625bb7)
(cherry picked from commit fed2b2daf5d44348a0096572e91708ac3db5e9c8)
(cherry picked from commit a9bafa50213a3a385ba3901b2730fb84149855cc)
(cherry picked from commit 8ce897b3fe918c5702d5c86d97f1703a1f266697)
(cherry picked from commit c806addcd07c6afb20d9aca833cfa9a33ac680a7)
(cherry picked from commit e70d51bd26b49de93b2fe012a4f03efbe6c97607)
(cherry picked from commit 57e597bf7e1e3bb3b7bcbcea66a4ea170a231f85)
(cherry picked from commit 643a2b0e81570e935779f6c509ebe4633fad74b9)
(cherry picked from commit f10faffb4febeef114d5be4e6abe57bd3cd72894)
(cherry picked from commit b440c5767eebdf406200e9a47446827778514425)
[TESTS] verify facts for the admin storage documentation (squash)
(cherry picked from commit d83d8ce57b8b39b4da849f5403198ecf706117ba)
(cherry picked from commit d8855ef27cd1b219184e95ce055bc6d84350ee26)
(cherry picked from commit 11230466ec0c1c4db1296cdd2ead74fc91a34491)
(cherry picked from commit b2cdd9d971b694fe32bab11f9ccdb41a38d7c6fe)
(cherry picked from commit a0a5e785241ac2c3a7493aa62637351021d48a39)
(cherry picked from commit 846413110fb936c386ba7fa80ff67e4394231464)
(cherry picked from commit 72b92d5a7854f5afbd949e0c6e53f3a4f5b72055)
(cherry picked from commit 7e039a9427d7a181ded17d653f49aa26679479e4)
(cherry picked from commit 227d42a1b69a1e542576ddc094d038de210183bb)
(cherry picked from commit 6488950a9b00b746c936fec41e6a9c59c4c02740)
(cherry picked from commit 0285c997749457cb6f67c5f3ca68e62721b53023)
(cherry picked from commit 0527bb20db0b22a1640f7d058399bc42ab8f9b98)
(cherry picked from commit b4d3fd43e314197f059830fe36d532888ef14c62)
(cherry picked from commit b45db6430024313f9530c1443d54630213f3b34b)
(cherry picked from commit 722ab376118847dca33ed5a47b6a41edda132b63)
(cherry picked from commit ff45632b42b1e5e741858a8f3bdb89a6d79cbe3e)
(cherry picked from commit 5144ae2aa7e1d903ecf6047902d9608cb6ae5500)
(cherry picked from commit f51438cd6888b9964c60369fecf64b727fe95ce1)
(cherry picked from commit 5ec304e5a178fb8c067b957233f54d8014df432f)
(cherry picked from commit 440b8c1af1d18f364f290fa385274d821b04c131)
(cherry picked from commit e5bbc626c1228a810bc8d6830cc5a3ee00d1baab)
(cherry picked from commit a996fe0fe4593d2f0432cbd8500f91cbbb011bb2)
(cherry picked from commit b7fe7cf401f4bddd6455efc651f7ac054f3fe1cf)
(cherry picked from commit cf339eed4f4851b18448dbdd83df32d00bc6f45b)
(cherry picked from commit 4f3a16168bbeced519a60c32e10e2895d9367238)
(cherry picked from commit 6f5bbc53fcebd614f5ee8627c7d6e3c637ffa694)
(cherry picked from commit aca42b422e76668387769c15868ef77b073bb7db)
(cherry picked from commit 5a7f7580e525694d1f27e12329c8532bd29273f8)
(cherry picked from commit 06c383c807ad49e1e35b429a10c6a11c65aeebe5)
(cherry picked from commit fe831dcb53b81b6cc632be751bfcbfb9bc00efd3)
(cherry picked from commit cd12cd0dbce47c6117ea579ae5019c182155b3a9)
(cherry picked from commit cc79163703ce31706c86b88c38fb8a20ed745e20)
(cherry picked from commit 0102a5715ea6a03b560cc2f0b6cbe2b2576c255e)
(cherry picked from commit 403f7520b3056eace36eae505afbab6a05f597b7)
(cherry picked from commit a3b61510a246f61c174ddc3c288e556522a6aab9)
(cherry picked from commit f83f0f9feb76a8b62ca4d74dc2785c713fbec282)
(cherry picked from commit fd1c3a6d09a057070844cd955d0920518e60f408)
(cherry picked from commit f7cdc3d6f1ad1ff01bb1814207fcb41210db80e8)
(cherry picked from commit 060121b644e0515a6b673cdd514d52e6fa1e6ec3)
(cherry picked from commit 62c847ff0235196e73e860d0d1658f0734985270)
(cherry picked from commit 4d051b51c2813828b978ef84b6b72c0ba051741b)
(cherry picked from commit 86e6981a936b1937065605892083b2488424bf5e)
(cherry picked from commit c1fc9e441b1467033911f3848f82a89aaacf98a9)
(cherry picked from commit 8bb2f0871a507d881acf74acf8fb90fd5ebff567)
(cherry picked from commit 0cd9fe52511ee06f1b8849325d196f60d8ccd151)
(cherry picked from commit b0b44778b4d3e51852d0669dfe6d9a6c334b90e4)
(cherry picked from commit 7c2f4f749f1c8986875eebf95254a3db151e5248)
Follow-up to d58c542579 for Forgejo.
By default, Gitea does not select any map service that can be used
to introduce a 'Show this place on a map' button in the location
field of a user profile. Before I tried upstreaming this change to
Gitea, this was the case in Forgejo. This patch essentially recovers
this functionality, which is nice for public-facing instances and
communities.
Links to original PRs:
- https://codeberg.org/forgejo/forgejo/pulls/1076
- https://github.com/go-gitea/gitea/pull/26214
(cherry picked from commit bb187d5f617f8efceb41810d6ff9adcaa60450bb)
(cherry picked from commit ce02ef9078a8731921caa4f7b0c1b0ac3b59a784)
(cherry picked from commit 6b75c40e2575e23810880ee8e368dc1781e2b4e4)
(cherry picked from commit 6bc8e9f5737f5721ddcd1ef5926a778a7f66a4a3)
(cherry picked from commit 063f8afdf7c2cca014c60dcc3d78d270fd236f8f)
(cherry picked from commit c5cc736b72e1ba980519e9258686f41e4eb78b42)
(cherry picked from commit 7b1bb4bedc6196fb6e8516d9a34092b16d7fab05)
(cherry picked from commit 2a022dceb4b391e2fc25b077193ae22a71380eee)
(cherry picked from commit a946c142d2448cb92daaea11851cf7f00921f062)
(cherry picked from commit 8a4ea0c7ab8960f41a67442fa2f485fe0a901a96)
(cherry picked from commit 37bfb05b34e61194afe407acac5778258315b756)
(cherry picked from commit 12fbbb1754a1a836f6923431e47a5de95aec6b2e)
(cherry picked from commit abe9de2cd91d7c5f176ceddff20ce194439703c4)
(cherry picked from commit 84db57871785d84c2a32edc22f769821b3bec961)
(cherry picked from commit 1beab7af463727ede3b2cc577696f8223fca78d1)
(cherry picked from commit 8779d505f2ab738e24503f74dd7bb6dfc8187996)
(cherry picked from commit 21a788f2eb549a494d52ab23a47533d9a7534ee6)
(cherry picked from commit a7cc6d168772e13bcf0808da14ff1ddf9baa64b2)
(cherry picked from commit 25f840379f19e0b90e69078e6c09a3c36fdf13ac)
(cherry picked from commit 0db65f11224f030877e5279b85af4b9e3e975ca7)
(cherry picked from commit 0e08cbc854cc1561101f1efe68de1d0860f7c905)
(cherry picked from commit 57de51db8d965a6c5a90477f58d2600fa3e7feda)
(cherry picked from commit 5ab3dcabb1461f4453f58718c6802baeffff06c9)
(cherry picked from commit 7b5ad0c13b44a705320965414bb4b366cb5a2a10)
(cherry picked from commit adf449a11595e0f16e6aceaeb0e7bfa277a6c333)
(cherry picked from commit 77aa18227e9f5d675ddae6b9f058f664b0d30795)
Renames it to `ENABLED` to be consistent with other settings and
deprecates it.
I believe this change is necessary because other setting groups such as
`attachment`, `cors`, `mailer`, etc. have an `ENABLED` setting, but
`oauth2` is the only one with an `ENABLE` setting, which could cause
confusion for users.
This is no longer a breaking change because `ENABLE` has been set as
deprecated and as an alias to `ENABLED`.
Sometimes you need to work on a feature which depends on another (unmerged) feature.
In this case, you may create a PR based on that feature instead of the main branch.
Currently, such PRs will be closed without the possibility to reopen in case the parent feature is merged and its branch is deleted.
Automatic target branch change make life a lot easier in such cases.
Github and Bitbucket behave in such way.
Example:
$PR_1$: main <- feature1
$PR_2$: feature1 <- feature2
Currently, merging $PR_1$ and deleting its branch leads to $PR_2$ being closed without the possibility to reopen.
This is both annoying and loses the review history when you open a new PR.
With this change, $PR_2$ will change its target branch to main ($PR_2$: main <- feature2) after $PR_1$ has been merged and its branch has been deleted.
This behavior is enabled by default but can be disabled.
For security reasons, this target branch change will not be executed when merging PRs targeting another repo.
Fixes#27062Fixes#18408
---------
Co-authored-by: Denys Konovalov <kontakt@denyskon.de>
Co-authored-by: delvh <dev.lh@web.de>
Mainly for MySQL/MSSQL.
It is important for Gitea to use case-sensitive database charset
collation. If the database is using a case-insensitive collation, Gitea
will show startup error/warning messages, and show the errors/warnings
on the admin panel's Self-Check page.
Make `gitea doctor convert` work for MySQL to convert the collations of
database & tables & columns.
* Fix#28131
## ⚠️ BREAKING ⚠️
It is not quite breaking, but it's highly recommended to convert the
database&table&column to a consistent and case-sensitive collation.
The CORS code has been unmaintained for long time, and the behavior is
not correct.
This PR tries to improve it. The key point is written as comment in
code. And add more tests.
Fix#28515Fix#27642Fix#17098
Nowadays, cache will be used on almost everywhere of Gitea and it cannot
be disabled, otherwise some features will become unaviable.
Then I think we can just remove the option for cache enable. That means
cache cannot be disabled.
But of course, we can still use cache configuration to set how should
Gitea use the cache.
## Changes
- Add deprecation warning to `Token` and `AccessToken` authentication
methods in swagger.
- Add deprecation warning header to API response. Example:
```
HTTP/1.1 200 OK
...
Warning: token and access_token API authentication is deprecated
...
```
- Add setting `DISABLE_QUERY_AUTH_TOKEN` to reject query string auth
tokens entirely. Default is `false`
## Next steps
- `DISABLE_QUERY_AUTH_TOKEN` should be true in a subsequent release and
the methods should be removed in swagger
- `DISABLE_QUERY_AUTH_TOKEN` should be removed and the implementation of
the auth methods in question should be removed
## Open questions
- Should there be further changes to the swagger documentation?
Deprecation is not yet supported for security definitions (coming in
[OpenAPI Spec version
3.2.0](https://github.com/OAI/OpenAPI-Specification/issues/2506))
- Should the API router logger sanitize urls that use `token` or
`access_token`? (This is obviously an insufficient solution on its own)
---------
Co-authored-by: delvh <dev.lh@web.de>
This patchset changes the connection string builder to use net.URL and
the host/port parser to use the stdlib function for splitting host from
port. It also adds a footnote about a potentially required portnumber
for postgres UNIX sockets.
Fixes: #24552
This PR adds a prefix path for all minio storage and override base path
will override the path.
The previous behavior is undefined officially, so it will be marked as
breaking.
Closes#27455
> The mechanism responsible for long-term authentication (the 'remember
me' cookie) uses a weak construction technique. It will hash the user's
hashed password and the rands value; it will then call the secure cookie
code, which will encrypt the user's name with the computed hash. If one
were able to dump the database, they could extract those two values to
rebuild that cookie and impersonate a user. That vulnerability exists
from the date the dump was obtained until a user changed their password.
>
> To fix this security issue, the cookie could be created and verified
using a different technique such as the one explained at
https://paragonie.com/blog/2015/04/secure-authentication-php-with-long-term-persistence#secure-remember-me-cookies.
The PR removes the now obsolete setting `COOKIE_USERNAME`.
Fix #27541
The INI package has a quirk: by default, the keys are inherited.
When maintaining the keys, the newly added sub key should not be
affected by the parent key.
Part of https://github.com/go-gitea/gitea/issues/27097:
- `gitea` theme is renamed to `gitea-light`
- `arc-green` theme is renamed to `gitea-dark`
- `auto` theme is renamed to `gitea-auto`
I put both themes in separate CSS files, removing all colors from the
base CSS. Existing users will be migrated to the new theme names. The
dark theme recolor will follow in a separate PR.
## ⚠️ BREAKING ⚠️
1. If there are existing custom themes with the names `gitea-light` or
`gitea-dark`, rename them before this upgrade and update the `theme`
column in the `user` table for each affected user.
2. The theme in `<html>` has moved from `class="theme-name"` to
`data-theme="name"`, existing customizations that depend on should be
updated.
---------
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
This pull request is a minor code cleanup.
From the Go specification (https://go.dev/ref/spec#For_range):
> "1. For a nil slice, the number of iterations is 0."
> "3. If the map is nil, the number of iterations is 0."
`len` returns 0 if the slice or map is nil
(https://pkg.go.dev/builtin#len). Therefore, checking `len(v) > 0`
before a loop is unnecessary.
---
At the time of writing this pull request, there wasn't a lint rule that
catches these issues. The closest I could find is
https://staticcheck.dev/docs/checks/#S103
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
This PR reduces the complexity of the system setting system.
It only needs one line to introduce a new option, and the option can be
used anywhere out-of-box.
It is still high-performant (and more performant) because the config
values are cached in the config system.
With this PR we added the possibility to configure the Actions timeouts
values for killing tasks/jobs.
Particularly this enhancement is closely related to the `act_runner`
configuration reported below:
```
# The timeout for a job to be finished.
# Please note that the Gitea instance also has a timeout (3h by default) for the job.
# So the job could be stopped by the Gitea instance if it's timeout is shorter than this.
timeout: 3h
```
---
Setting the corresponding key in the INI configuration file, it is
possible to let jobs run for more than 3 hours.
Signed-off-by: Francesco Antognazza <francesco.antognazza@gmail.com>
If the AppURL(ROOT_URL) is an HTTPS URL, then the COOKIE_SECURE's
default value should be true.
And, if a user visits an "http" site with "https" AppURL, they won't be
able to login, and they should have been warned. The only problem is
that the "language" can't be set either in such case, while I think it
is not a serious problem, and it could be fixed easily if needed.
This feature was removed by #22219 to avoid possible CSRF attack.
This PR takes reverseproxy auth for API back but with default disabled.
To prevent possbile CSRF attack, the responsibility will be the
reverseproxy but not Gitea itself.
For those want to enable this `ENABLE_REVERSE_PROXY_AUTHENTICATION_API`,
they should know what they are doing.
---------
Co-authored-by: Giteabot <teabot@gitea.io>
Currently, Artifact does not have an expiration and automatic cleanup
mechanism, and this feature needs to be added. It contains the following
key points:
- [x] add global artifact retention days option in config file. Default
value is 90 days.
- [x] add cron task to clean up expired artifacts. It should run once a
day.
- [x] support custom retention period from `retention-days: 5` in
`upload-artifact@v3`.
- [x] artifacts link in actions view should be non-clickable text when
expired.
This PR rewrites the function `getStorage` and make it more clear.
Include tests from #26435, thanks @earl-warren
---------
Co-authored-by: Earl Warren <contact@earl-warren.org>
When users put the secrets into a file (GITEA__sec__KEY__FILE), the
newline sometimes is different to avoid (eg: echo/vim/...)
So the last newline could be removed when reading, it makes the users
easier to maintain the secret files.
Co-authored-by: Giteabot <teabot@gitea.io>
This PR is an extended implementation of #25189 and builds upon the
proposal by @hickford in #25653, utilizing some ideas proposed
internally by @wxiaoguang.
Mainly, this PR consists of a mechanism to pre-register OAuth2
applications on startup, which can be enabled or disabled by modifying
the `[oauth2].DEFAULT_APPLICATIONS` parameter in app.ini. The OAuth2
applications registered this way are being marked as "locked" and
neither be deleted nor edited over UI to prevent confusing/unexpected
behavior. Instead, they're being removed if no longer enabled in config.
The implemented mechanism can also be used to pre-register other OAuth2
applications in the future, if wanted.
Co-authored-by: hickford <mirth.hickford@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
---------
Co-authored-by: M Hickford <mirth.hickford@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
