(cherry picked from commit b7fe7cf401f4bddd6455efc651f7ac054f3fe1cf)
(cherry picked from commit cf339eed4f4851b18448dbdd83df32d00bc6f45b)
(cherry picked from commit 4f3a16168bbeced519a60c32e10e2895d9367238)
(cherry picked from commit 6f5bbc53fcebd614f5ee8627c7d6e3c637ffa694)
(cherry picked from commit aca42b422e76668387769c15868ef77b073bb7db)
(cherry picked from commit 5a7f7580e525694d1f27e12329c8532bd29273f8)
(cherry picked from commit 06c383c807ad49e1e35b429a10c6a11c65aeebe5)
(cherry picked from commit fe831dcb53b81b6cc632be751bfcbfb9bc00efd3)
(cherry picked from commit cd12cd0dbce47c6117ea579ae5019c182155b3a9)
(cherry picked from commit cc79163703ce31706c86b88c38fb8a20ed745e20)
(cherry picked from commit 0102a5715ea6a03b560cc2f0b6cbe2b2576c255e)
(cherry picked from commit 403f7520b3056eace36eae505afbab6a05f597b7)
(cherry picked from commit a3b61510a246f61c174ddc3c288e556522a6aab9)
(cherry picked from commit f83f0f9feb76a8b62ca4d74dc2785c713fbec282)
(cherry picked from commit fd1c3a6d09a057070844cd955d0920518e60f408)
(cherry picked from commit f7cdc3d6f1ad1ff01bb1814207fcb41210db80e8)
(cherry picked from commit 060121b644e0515a6b673cdd514d52e6fa1e6ec3)
(cherry picked from commit 62c847ff0235196e73e860d0d1658f0734985270)
(cherry picked from commit 4d051b51c2813828b978ef84b6b72c0ba051741b)
(cherry picked from commit 86e6981a936b1937065605892083b2488424bf5e)
(cherry picked from commit c1fc9e441b1467033911f3848f82a89aaacf98a9)
(cherry picked from commit 8bb2f0871a507d881acf74acf8fb90fd5ebff567)
(cherry picked from commit 0cd9fe52511ee06f1b8849325d196f60d8ccd151)
(cherry picked from commit b0b44778b4d3e51852d0669dfe6d9a6c334b90e4)
(cherry picked from commit 7c2f4f749f1c8986875eebf95254a3db151e5248)
Follow-up to d58c542579 for Forgejo.
By default, Gitea does not select any map service that can be used
to introduce a 'Show this place on a map' button in the location
field of a user profile. Before I tried upstreaming this change to
Gitea, this was the case in Forgejo. This patch essentially recovers
this functionality, which is nice for public-facing instances and
communities.
Links to original PRs:
- https://codeberg.org/forgejo/forgejo/pulls/1076
- https://github.com/go-gitea/gitea/pull/26214
(cherry picked from commit bb187d5f617f8efceb41810d6ff9adcaa60450bb)
(cherry picked from commit ce02ef9078a8731921caa4f7b0c1b0ac3b59a784)
(cherry picked from commit 6b75c40e2575e23810880ee8e368dc1781e2b4e4)
(cherry picked from commit 6bc8e9f5737f5721ddcd1ef5926a778a7f66a4a3)
(cherry picked from commit 063f8afdf7c2cca014c60dcc3d78d270fd236f8f)
(cherry picked from commit c5cc736b72e1ba980519e9258686f41e4eb78b42)
(cherry picked from commit 7b1bb4bedc6196fb6e8516d9a34092b16d7fab05)
(cherry picked from commit 2a022dceb4b391e2fc25b077193ae22a71380eee)
(cherry picked from commit a946c142d2448cb92daaea11851cf7f00921f062)
(cherry picked from commit 8a4ea0c7ab8960f41a67442fa2f485fe0a901a96)
(cherry picked from commit 37bfb05b34e61194afe407acac5778258315b756)
(cherry picked from commit 12fbbb1754a1a836f6923431e47a5de95aec6b2e)
(cherry picked from commit abe9de2cd91d7c5f176ceddff20ce194439703c4)
(cherry picked from commit 84db57871785d84c2a32edc22f769821b3bec961)
(cherry picked from commit 1beab7af463727ede3b2cc577696f8223fca78d1)
(cherry picked from commit 8779d505f2ab738e24503f74dd7bb6dfc8187996)
(cherry picked from commit 21a788f2eb549a494d52ab23a47533d9a7534ee6)
(cherry picked from commit a7cc6d168772e13bcf0808da14ff1ddf9baa64b2)
(cherry picked from commit 25f840379f19e0b90e69078e6c09a3c36fdf13ac)
(cherry picked from commit 0db65f11224f030877e5279b85af4b9e3e975ca7)
(cherry picked from commit 0e08cbc854cc1561101f1efe68de1d0860f7c905)
(cherry picked from commit 57de51db8d965a6c5a90477f58d2600fa3e7feda)
(cherry picked from commit 5ab3dcabb1461f4453f58718c6802baeffff06c9)
(cherry picked from commit 7b5ad0c13b44a705320965414bb4b366cb5a2a10)
(cherry picked from commit adf449a11595e0f16e6aceaeb0e7bfa277a6c333)
(cherry picked from commit 77aa18227e9f5d675ddae6b9f058f664b0d30795)
(cherry picked from commit 3ea0b287d74b8fc0dad08b2a539105e1aa1c1e67)
(cherry picked from commit db8392a8ac093d4d3760e8bb40c56d8e194d44fb)
(cherry picked from commit bd2a5fa2923c320e01faeaa1fdc1ad823c337027)
(cherry picked from commit 2436acb3d986bad08aa134e450420fc4a08f5d62)
(cherry picked from commit 62f50e1c521c30729781e5cd58882e743bb8b851)
(cherry picked from commit dba18925217254d22f76306c0fe1c046c419268e)
(cherry picked from commit 4b58e3b6d41f9e42fd34469184a3c7d4c5205c86)
(cherry picked from commit 12470568569fec0644319d3d263a82bc7acdc0c3)
(cherry picked from commit afbaea700972ddd4cb788d0b6d5e78de5558756a)
(cherry picked from commit dcd4813d96f8d4e429914f4c8f951bd25b23afca)
(cherry picked from commit b51dc963d1625bd7b9869302ec1ae70bbafb4442)
(cherry picked from commit 611e895efd28026fdc4661893172356e76f6c3d0)
(cherry picked from commit fd492a03f5335c138e143e784657f7012484bd8c)
(cherry picked from commit 2c99991f44a15466339bb82b29d455c66795721b)
(cherry picked from commit 7426c1edb4b904424cded473c94c9739fde83ed0)
(cherry picked from commit 373244f8b2d449e8811e488e3a3732ba9bc79eaa)
(cherry picked from commit 4f6efecdb9d932459dd9f4d7ee393a121a7bbb96)
(cherry picked from commit 61d500808e443ce770b06fa1b87728a7785bd387)
(cherry picked from commit 65f8384b6361c39d48df95227b047a20977482be)
(cherry picked from commit 12ed28e734f79203ad6bf94774008715da6efd21)
(cherry picked from commit ec6cdc9e1a9544f4f2b0ad7256ed2a8b15cf5335)
(cherry picked from commit 08653ba05119906e066455a3e3913e66c179dbf3)
(cherry picked from commit d5847c87cbcf0fd15953fb7aeadd1cece989b6c9)
(cherry picked from commit 640a96e19be0c7faedf5aedfd90c15df8dc8f376)
(cherry picked from commit 46177814a9ee8596b58055b57bea5441a893de7f)
(cherry picked from commit b0098f5a80356757107cc4dd58bf24e2acba6b26)
(cherry picked from commit ce5ddeeca9fa39180a2f9630c82007f8e9410867)
(cherry picked from commit 5736fa1025681244b8eefef9a14b0715fa37d9a1)
(cherry picked from commit c43ca210fcbc2589158df8c2c3036dedb00eaa52)
(cherry picked from commit 7f92906bf3c72649dd2668263761ded71cf5ee91)
(cherry picked from commit f726525d2dba5e481f758624e62eeaa3d38be564)
(cherry picked from commit db86c93b0b7d2c1a7c17fcb6047b3d75873f9bbe)
(cherry picked from commit 6751bd93c3faf4be0f29b52b4c41626a244a2a54)
(cherry picked from commit 74bb523ac96062f93f20c174dcc9b1dc7ad94b13)
(cherry picked from commit 94f9045a81ef58ecb6671d1e8f2ad31a7758ea9a)
(cherry picked from commit 5297eac42d905d10060de688963f111cbefe49d4)
(cherry picked from commit 57e3c57c519f41a595845be3f2505610ae8ad690)
(cherry picked from commit c5cacfee51e2e7846234c9dd9c053b6cdfaf3947)
(cherry picked from commit dfa31ee0048dcf5c3b8d6fec6184f665a019f014)
(cherry picked from commit d7d10a76b41079cab423f00a96426a8de19fb876)
(cherry picked from commit 62bd4edd4622ea778d8994d05c535de677819544)
(cherry picked from commit 798c211f86f4ce713ab3fc18411fd72a57a219ef)
(cherry picked from commit 1f645aeceafdf6f4f74864b14d939c4bcdf096be)
(cherry picked from commit 8a8b62e10e9678fe33e32fa75f4b8c78e6aef68f)
(cherry picked from commit d3ff4e1fdfdaa272941d86735c91107f03280294)
(cherry picked from commit 81412571f8daeb4797556ba846ef7d7280844ef2)
(cherry picked from commit e9faa1f4e02c567721d2c28426580a839b96de5e)
(cherry picked from commit bce1ab85f7cef1696a4560f079bc346e813814b2)
(cherry picked from commit f9d75d4705ece5c119f2fd5e2bfbaf630d0bf739)
(cherry picked from commit 64f76f4ab269daa6a584305164732c68be0161c5)
(cherry picked from commit 5d024541551f1be98444923f0fa327a855104725)
[CI] Forgejo Actions workflows
(cherry picked from commit 3ff59b5379ebf761f32875f9d869a1d18f79741c)
(cherry picked from commit 8af826a6f7c70d03079ec68f10230041695017cd)
(cherry picked from commit d7c09d9cc80037a28d488da142ae2a2a99f59ac9)
[CI] use the docker label instead of ubuntu-latest
(cherry picked from commit b6a6470db6c8bae9963e204f9c8c408f309e81e3)
[CI] all tests need compliance before proceeding
(cherry picked from commit b35c496f2c3034164b9bb3a3550f35026adf9372)
(cherry picked from commit 36a4148a8ec654ae9fa5a6925a3b8606b96aebcc)
(cherry picked from commit 7ffcffa653808a284f422fdc31f6ea07874b585d)
(cherry picked from commit 8a246d296e502c181c6ee779d1773c9e78ca7acf)
(cherry picked from commit dd0b6e1826ba7699f967de2c00c1332909a53473)
[CI] Forgejo Actions based release process (squash) MySQL optimization
Refs: https://codeberg.org/forgejo/forgejo/issues/976
(cherry picked from commit b4b8c489e6e9fe36349eced5d4249467b25df2a7)
(cherry picked from commit 1e861db4afdc32549784381a96b16dac508ec1c0)
(cherry picked from commit a6c0e00330ded851c2f524039259f0b723d793e4)
(cherry picked from commit f97b336465fd6be15079e756c9ee3a580f0aa339)
(cherry picked from commit 6d65d5f0d6f798556a8f9e547896be03a5ee2f87)
(cherry picked from commit 79bfbadbed3bf84b96f2c027d119a4aa5c60d5e2)
(cherry picked from commit e86c40a34a97fc568c724ee5f8ee5b2bf46ebf19)
(cherry picked from commit da0c454adbcf1b5dbb95d9d74afaa371e32351a0)
(cherry picked from commit b49d892cda968b22ad6c29caade042690d96f2e2)
[CI] enable minio tests
(cherry picked from commit 4d8f4380311b872061bd20f561424662ecd8d817)
(cherry picked from commit c4eeb0a61ef579c21b0382edd61c62adebe93b28)
[CI] Forgejo Actions based CI for PR & branches (squash) cleanup
(cherry picked from commit 80eb20e84267552c971096693558734f0ec7afdd)
(cherry picked from commit d2ff589858961e4f25bc1d903cdf57df1db4c316)
(cherry picked from commit f6eedecb67fce992605bb1fcdd93c866ec47a9d4)
(cherry picked from commit cf458091e245e1c70944e940a214d5127f304303)
(cherry picked from commit ddd322cb2de165126080b222c1b09b2a91fea182)
(cherry picked from commit f0f5729b6467bea053d532c4c64c54a0fe737c98)
Conflicts:
.github/workflows/pull-db-tests.yml
https://codeberg.org/forgejo/forgejo/pulls/1573
(cherry picked from commit bb347aedd4e5030eea46ce22176a0444619df6b8)
(cherry picked from commit 0f5ebe8c3e78cef6f310ee9097a8413df904c08a)
(cherry picked from commit 9101cb4715ed88c6a3f0ad14443758d9bc9c61ed)
(cherry picked from commit ab118fe4efd96e9bb7134065643f094a379bb192)
[CI] Forgejo Actions based CI for PR & branches (squash) use node:20-bookworm
No longer use the custom test-env image, it is unecessary technical
debt.
Also upgrade to bitnami/minio:2023.8.31 to align with what Gitea tests
(cherry picked from commit d9b77fd2735a52043b4f8f1baaaa2e15073db621)
(cherry picked from commit ef8f099f9ea03d216324ceff348fe643240341dc)
(cherry picked from commit 6d3c675d203ba838de1f0540977f5e7e9f62f055)
(cherry picked from commit fc00ff45f2e810cc2a84bf57cdb2765830e5b07f)
(cherry picked from commit b7f02b9846b8c7087a4827a9c87f48eafd2ba1a2)
(cherry picked from commit d23d86e56728f4a263a0567c14e2dc20407baaff)
(cherry picked from commit e68e65460c40d343a3ada858c0c48d03a6a14bbb)
(cherry picked from commit 96f93a1fdb9f2c6ecbd05ac24c5375f3e0d631df)
(cherry picked from commit 2ff3080018633212075935ec6644654184009a2e)
(cherry picked from commit 8ea00bcff471b877e442f7b78bb46880c9e396f0)
(cherry picked from commit 85be961970b0d374d9816348a8a1b51289f7c787)
(cherry picked from commit 387b279fc33d79be1f0f0b35b32f5c17b0d64c9c)
(cherry picked from commit 219be7aa85fbf2bd5cb9daead3544b27f6200d83)
(cherry picked from commit 929b75ee39cd6d1a462bbd0c20597c357949c953)
(cherry picked from commit dd42a30d6a9e6cad26c9c916d733a0270d94a87e)
(cherry picked from commit 8b11cab677503be78b1deb17ed9dd5fb1c823a7d)
(cherry picked from commit be5927069674a17a4c09e7f0aa530bc4630851a9)
(cherry picked from commit e068f8b191585e2910d8a45ea78bfa1b78015bed)
(cherry picked from commit 7855bb0c60b5ec2a972ae04e4515ee5adb19a5e7)
(cherry picked from commit 45c4c8f44383dced75ab83f7c817b52e78968fab)
(cherry picked from commit 89520d67ffe0062e1accd39763e1e7dd5058d83a)
(cherry picked from commit 15eeb417a4b8bb948f888c73e20135c1e0fd1f63)
(cherry picked from commit 6db53a26432f02ae50da948483e2010bd962f9ce)
(cherry picked from commit 2f689b321fa275b6412f0b8686edc7aba97c3565)
(cherry picked from commit 04dc478314c3b4927cca78c354ca46ee217f035a)
(cherry picked from commit a554624f40f51c1c75d754d6cca14f7626bb599e)
(cherry picked from commit abca05f0d1c29680bba897cc0de7037053915ced)
(cherry picked from commit dc13e7eb22f2bde817f3845c646574d8f39c1b18)
(cherry picked from commit a161c5740eb5e76c13354ba2388fa34ae925fd8b)
(cherry picked from commit 06d33e2773f01b576ba050afe2b88718a7999434)
(cherry picked from commit f536275161cc4bf5d2f163bd68a4c4498c9fff3d)
(cherry picked from commit 84ac6f314a1840d45bdecb2ccd4482fb925fd2c2)
(cherry picked from commit 1e8126edfc3a6c78cda35f053bd0ac40ba9874ef)
(cherry picked from commit 0287ac3416563e0af05c3aeabc338b791c74ddb2)
(cherry picked from commit 3e5fca2aaea299013691f102b10417ac33988df0)
(cherry picked from commit 03b220bfeb86caa82eec2a67caf9a08460cf76c0)
(cherry picked from commit 1d033f4aaf2b7db1ae98e91d96ef3b8a0b07539e)
(cherry picked from commit 2ee9e3e9a395357885f16fa8a22a24cda79f637f)
(cherry picked from commit d28c2849931e7de2ae1660513774047bf6868e1a)
(cherry picked from commit 239df83859f88f7833f5796ee1f0811732c6e9a2)
(cherry picked from commit 96ae0c2e5d4fafbead44db52297e21242da0a6d0)
(cherry picked from commit 49aef71b322395674a5360bac7b93561e773ea35)
(cherry picked from commit 38b56d108d3f27faa0dc191518e6dbcb775fd7bf)
(cherry picked from commit 30f8d9ec3adecf4025fc3547e9a745d700f68d83)
(cherry picked from commit d5318618509f5d8bdef999445357553ab1c4cb5c)
(cherry picked from commit a75707deaa52cd3fedfda766460bcc8f2d7dea92)
(cherry picked from commit b1c73918b2ddaa4a59d9e7405b3b5bea5ccd496a)
(cherry picked from commit 53919170919bfcd3dca00ecbeb0778eadb0af075)
(cherry picked from commit a427f8dae539e9b38b8ca0e86ebb09cc2940e5a7)
(cherry picked from commit 6ba6f62c7e696b6975fbd797b1c596ff28c64e91)
(cherry picked from commit a1381d9146fba42cb97d72d38525fa3e721bfb03)
(cherry picked from commit 74714e02461fb47fcc0901211668e4529fac68d0)
(cherry picked from commit 7749dbfe6684498a47e3037088e7bef3542b6ce5)
(cherry picked from commit 437924971136eaed795f77edd3d3dfffa5f68103)
(cherry picked from commit a69f55bebf82a0b68bc0f66bc029eaea836cddb7)
(cherry picked from commit 24dd5fbfdbc27c887dbc24661c1005fb2e14e3c6)
(cherry picked from commit dda856d6b83936fd1c96c84544b086cbd8f63115)
(cherry picked from commit bc14f4fa97fffe82d1c666e961e313f88433cb9e)
(cherry picked from commit 78fef4f1379d8854901151d4bc62135c73db868e)
(cherry picked from commit 69e013cc515e2a50006d8d02f575ff6490d272ff)
(cherry picked from commit f173c6a2734b2dccf1424d27cd8e10fc296e44a4)
(cherry picked from commit 92f9d02547017770deafd1f715c32ae4479b8ded)
(cherry picked from commit c99d51e665370ceb71b96b3fb65184090c7e4442)
(cherry picked from commit aa0650fd2b42738a5e564c229c3eb63b8ca77f9b)
(cherry picked from commit 0a8ef91302368751df22a1967857283222bc097f)
(cherry picked from commit 7b54fe01c2ded0bbbcae6b89d9e330ca4f6ab744)
(cherry picked from commit 0e154f366f14d106d14f500f605380c29b5a3f21)
(cherry picked from commit 02d88ee16d23b9ebb04bf1af843fc5d2074783ce)
(cherry picked from commit 411924e0172a7b10de7513f2e7f60ab5341b13e4)
(cherry picked from commit f4e9ca6db59f2c5c638a0560d4ea99833d61520b)
(cherry picked from commit cd80126a23573dd5aea1e9674ee0bfa34c63ec5a)
(cherry picked from commit da626702f9743fc6e1dd77d21aff5fc3afe75912)
(cherry picked from commit 4b81d0bd046fef267bb10d2ca0cbd342c87fd4e2)
(cherry picked from commit 53ac2606694fa060879a0f4c82f6164c6f42a4d0)
(cherry picked from commit 984081f08d108acc47d312307b1c3beee3058202)
(cherry picked from commit 1c39bae3ec9b485f9969e29ed7ae8fe37b32da69)
(cherry picked from commit e1bbfa36197ebab97954e8195f7d36adf7c85d56)
(cherry picked from commit 91245ca9179a46047a351247dacecdace557111d)
(cherry picked from commit 705d0558be2c90d06e9e5b883044fd0b275b1113)
(cherry picked from commit 9247594970c9db109e3e6ca3fd87485450df921c)
(cherry picked from commit 9db1158a487e00e588810459fe402cc2ccea43f7)
(cherry picked from commit 3b36b77d87a90fbea03fc16638657e19328ccedc)
(cherry picked from commit 162fa1d8ae3753dd8ee51698555e495f2c63d925)
(cherry picked from commit d03d0afbb565c8bc8b723e10c8c70b69e7af7b80)
(cherry picked from commit 7b8f92f7871b838bc2eefa34e7dc48bcd141d1d5)
(cherry picked from commit 035abca9691d33e319062325dae402da66683c43)
(cherry picked from commit a8fbf6bb56046665cb2cde0ffcc753f56b2f0f2d)
(cherry picked from commit 3be681d037b07880236cae1aa70245e5eb4d1497)
(cherry picked from commit 7e5d471c832ee3fea378ecc97835b038bd55a8e1)
(cherry picked from commit 323801d935fec2c6d460192b62fa12b5204da76e)
(cherry picked from commit 3fdfe4bfea623111f1f97e50b71b98a63c8b38e7)
(cherry picked from commit 58a07421a4508ca298c1c3a45d33d49737ee98d8)
(cherry picked from commit dbb71a4c8502b640857d3500dda12ab4b5d74b29)
(cherry picked from commit d442113520d21149e155d1e62bbeb6a35a6aec08)
(cherry picked from commit d3329f01f8c7145c422b159509f544ec83604a51)
(cherry picked from commit 069a1d68b856898e2913d1d4456deb7f1e976a6c)
(cherry picked from commit 14919e609a4dd9ae9ca19880ffc459def8bea273)
(cherry picked from commit 49b76be1068d1f83169956bb141116481a7e6a3c)
(cherry picked from commit 0fe9f257d2bd277f5cd620fe04e4b80b5abcd585)
(cherry picked from commit b583bebeab3d0b182df6b5d087522a4fb89ba3e9)
(cherry picked from commit 5c616e43a64451d607b6ee24400708d2704fd4db)
(cherry picked from commit 854bcea9051dc615cfd6d3e8cb03986e9058fd65)
(cherry picked from commit c2acb181c57e6ffef37df1a3a3b1b63c326cdd43)
(cherry picked from commit 1cb07e71d14118871ae40a82adabcde851a3e172)
(cherry picked from commit 5d3f09e6351614a8db979995299ac1b94ebf08ee)
(cherry picked from commit f8bf1c8d42be0eb40f4d4fdc72e7e4cefa842e52)
(cherry picked from commit a471ed4576607a4e13cac980016c9e2a702d9fd5)
(cherry picked from commit 95c755f4e34bb753a0f94f87f02a17256d7d1619)
(cherry picked from commit 1d8bc5215f6918e11d8beb1f7e252b04d9c15bb6)
(cherry picked from commit 45c1e7b8d0920db98556ecfdf0d1111c2ffcb66e)
(cherry picked from commit 2eb4d93af7b5679228dc38578a746242250e5d92)
(cherry picked from commit 98dbce5e147432194d6f177133dcaabe04309712)
(cherry picked from commit fbe2fb5861c90fcc292f357f45f804ee87594b6e)
Resolves https://github.com/go-gitea/gitea/issues/28704
Example of an entry in the generated `APKINDEX` file:
```
C:Q1xCO3H9LTTEbhKt9G1alSC87I56c=
P:hello
V:2.12-r1
A:x86_64
T:The GNU Hello program produces a familiar, friendly greeting
U:https://www.gnu.org/software/hello/
L:GPL-3.0-or-later
S:15403
I:36864
o:hello
m:
t:1705934118
D:so:libc.musl-x86_64.so.1
p:cmd:hello=2.12-r1
i:foobar=1.0 !baz
k:42
```
the `i:` and `k:` entries are new.
---------
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Fixes#28660
Fixes an admin api bug related to `user.LoginSource`
Fixed `/user/emails` response not identical to GitHub api
This PR unifies the user update methods. The goal is to keep the logic
only at one place (having audit logs in mind). For example, do the
password checks only in one method not everywhere a password is updated.
After that PR is merged, the user creation should be next.
In #28691, schedule plans will be deleted when a repo's actions unit is
disabled. But when the unit is enabled, the schedule plans won't be
created again.
This PR fixes the bug. The schedule plans will be created again when the
actions unit is re-enabled
Renames it to `ENABLED` to be consistent with other settings and
deprecates it.
I believe this change is necessary because other setting groups such as
`attachment`, `cors`, `mailer`, etc. have an `ENABLED` setting, but
`oauth2` is the only one with an `ENABLE` setting, which could cause
confusion for users.
This is no longer a breaking change because `ENABLE` has been set as
deprecated and as an alias to `ENABLED`.
## Purpose
This is a refactor toward building an abstraction over managing git
repositories.
Afterwards, it does not matter anymore if they are stored on the local
disk or somewhere remote.
## What this PR changes
We used `git.OpenRepository` everywhere previously.
Now, we should split them into two distinct functions:
Firstly, there are temporary repositories which do not change:
```go
git.OpenRepository(ctx, diskPath)
```
Gitea managed repositories having a record in the database in the
`repository` table are moved into the new package `gitrepo`:
```go
gitrepo.OpenRepository(ctx, repo_model.Repo)
```
Why is `repo_model.Repository` the second parameter instead of file
path?
Because then we can easily adapt our repository storage strategy.
The repositories can be stored locally, however, they could just as well
be stored on a remote server.
## Further changes in other PRs
- A Git Command wrapper on package `gitrepo` could be created. i.e.
`NewCommand(ctx, repo_model.Repository, commands...)`. `git.RunOpts{Dir:
repo.RepoPath()}`, the directory should be empty before invoking this
method and it can be filled in the function only. #28940
- Remove the `RepoPath()`/`WikiPath()` functions to reduce the
possibility of mistakes.
---------
Co-authored-by: delvh <dev.lh@web.de>
The `ToUTF8*` functions were stripping BOM, while BOM is actually valid
in UTF8, so the stripping must be optional depending on use case. This
does:
- Add a options struct to all `ToUTF8*` functions, that by default will
strip BOM to preserve existing behaviour
- Remove `ToUTF8` function, it was dead code
- Rename `ToUTF8WithErr` to `ToUTF8`
- Preserve BOM in Monaco Editor
- Remove a unnecessary newline in the textarea value. Browsers did
ignore it, it seems but it's better not to rely on this behaviour.
Fixes: https://github.com/go-gitea/gitea/issues/28743
Related: https://github.com/go-gitea/gitea/issues/6716 which seems to
have once introduced a mechanism that strips and re-adds the BOM, but
from what I can tell, this mechanism was removed at some point after
that PR.
Fix#22066
# Purpose
This PR fix the releases will be deleted when mirror repository sync the
tags.
# The problem
In the previous implementation of #19125. All releases record in
databases of one mirror repository will be deleted before sync.
Ref:
https://github.com/go-gitea/gitea/pull/19125/files#diff-2aa04998a791c30e5a02b49a97c07fcd93d50e8b31640ce2ddb1afeebf605d02R481
# The Pros
This PR introduced a new method which will load all releases from
databases and all tags on git data into memory. And detect which tags
needs to be inserted, which tags need to be updated or deleted. Only
tags releases(IsTag=true) which are not included in git data will be
deleted, only tags which sha1 changed will be updated. So it will not
delete any real releases include drafts.
# The Cons
The drawback is the memory usage will be higher than before if there are
many tags on this repository. This PR defined a special release struct
to reduce columns loaded from database to memory.
This should fix https://github.com/go-gitea/gitea/issues/28927
Technically older versions of Git would support this flag as well, but
per https://github.com/go-gitea/gitea/pull/28466 that's the version
where using it (object-format=sha256) left "experimental" state.
`sha1` is (currently) the default, so older clients should be unaffected
in either case.
Signed-off-by: jolheiser <john.olheiser@gmail.com>
When LFS hooks are present in gitea-repositories, operations like git
push for creating a pull request fail. These repositories are not meant
to include LFS files or git push them, that is handled separately. And
so they should not have LFS hooks.
Installing git-lfs on some systems (like Debian Linux) will
automatically set up /etc/gitconfig to create LFS hooks in repositories.
For most git commands in Gitea this is not a problem, either because
they run on a temporary clone or the git command does not create LFS
hooks.
But one case where this happens is git archive for creating repository
archives. To fix that, add a GIT_CONFIG_NOSYSTEM=1 to disable using the
system configuration for that command.
According to a comment, GIT_CONFIG_NOSYSTEM is not used for all git
commands because the system configuration can be intentionally set up
for Gitea to use.
Resolves#19810, #21148
Sometimes you need to work on a feature which depends on another (unmerged) feature.
In this case, you may create a PR based on that feature instead of the main branch.
Currently, such PRs will be closed without the possibility to reopen in case the parent feature is merged and its branch is deleted.
Automatic target branch change make life a lot easier in such cases.
Github and Bitbucket behave in such way.
Example:
$PR_1$: main <- feature1
$PR_2$: feature1 <- feature2
Currently, merging $PR_1$ and deleting its branch leads to $PR_2$ being closed without the possibility to reopen.
This is both annoying and loses the review history when you open a new PR.
With this change, $PR_2$ will change its target branch to main ($PR_2$: main <- feature2) after $PR_1$ has been merged and its branch has been deleted.
This behavior is enabled by default but can be disabled.
For security reasons, this target branch change will not be executed when merging PRs targeting another repo.
Fixes#27062Fixes#18408
---------
Co-authored-by: Denys Konovalov <kontakt@denyskon.de>
Co-authored-by: delvh <dev.lh@web.de>
Fixes#26548
This PR refactors the rendering of markup links. The old code uses
`strings.Replace` to change some urls while the new code uses more
context to decide which link should be generated.
The added tests should ensure the same output for the old and new
behaviour (besides the bug).
We may need to refactor the rendering a bit more to make it clear how
the different helper methods render the input string. There are lots of
options (resolve links / images / mentions / git hashes / emojis / ...)
but you don't really know what helper uses which options. For example,
we currently support images in the user description which should not be
allowed I think:
<details>
<summary>Profile</summary>
https://try.gitea.io/KN4CK3R
</details>
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Fixes#27114.
* In Gitea 1.12 (#9532), a "dismiss stale approvals" branch protection
setting was introduced, for ignoring stale reviews when verifying the
approval count of a pull request.
* In Gitea 1.14 (#12674), the "dismiss review" feature was added.
* This caused confusion with users (#25858), as "dismiss" now means 2
different things.
* In Gitea 1.20 (#25882), the behavior of the "dismiss stale approvals"
branch protection was modified to actually dismiss the stale review.
For some users this new behavior of dismissing the stale reviews is not
desirable.
So this PR reintroduces the old behavior as a new "ignore stale
approvals" branch protection setting.
---------
Co-authored-by: delvh <dev.lh@web.de>
Fix#28157
This PR fix the possible bugs about actions schedule.
## The Changes
- Move `UpdateRepositoryUnit` and `SetRepoDefaultBranch` from models to
service layer
- Remove schedules plan from database and cancel waiting & running
schedules tasks in this repository when actions unit has been disabled
or global disabled.
- Remove schedules plan from database and cancel waiting & running
schedules tasks in this repository when default branch changed.
Mainly for MySQL/MSSQL.
It is important for Gitea to use case-sensitive database charset
collation. If the database is using a case-insensitive collation, Gitea
will show startup error/warning messages, and show the errors/warnings
on the admin panel's Self-Check page.
Make `gitea doctor convert` work for MySQL to convert the collations of
database & tables & columns.
* Fix#28131
## ⚠️ BREAKING ⚠️
It is not quite breaking, but it's highly recommended to convert the
database&table&column to a consistent and case-sensitive collation.
In #26365 issue references were disabled entirely for documents,
intending to match GitHub behavior. However cross-references do appear
to work in documents on GitHub.
This is useful for example to write release notes in a markdown document
and reference issues. While the simpler syntax may create links when not
intended, hopefully the cross-reference syntax is unique enough to avoid
it.
The CORS code has been unmaintained for long time, and the behavior is
not correct.
This PR tries to improve it. The key point is written as comment in
code. And add more tests.
Fix#28515Fix#27642Fix#17098
Fix#28526, regression of
* #26365
(although the author of #26365 has recent activities, but there is no
response for the regression, so I proposed this quick fix and keep the
fix simple to make it easier to backport to 1.21)
Nowadays, cache will be used on almost everywhere of Gitea and it cannot
be disabled, otherwise some features will become unaviable.
Then I think we can just remove the option for cache enable. That means
cache cannot be disabled.
But of course, we can still use cache configuration to set how should
Gitea use the cache.
The 4 functions are duplicated, especially as interface methods. I think
we just need to keep `MustID` the only one and remove other 3.
```
MustID(b []byte) ObjectID
MustIDFromString(s string) ObjectID
NewID(b []byte) (ObjectID, error)
NewIDFromString(s string) (ObjectID, error)
```
Introduced the new interfrace method `ComputeHash` which will replace
the interface `HasherInterface`. Now we don't need to keep two
interfaces.
Reintroduced `git.NewIDFromString` and `git.MustIDFromString`. The new
function will detect the hash length to decide which objectformat of it.
If it's 40, then it's SHA1. If it's 64, then it's SHA256. This will be
right if the commitID is a full one. So the parameter should be always a
full commit id.
@AdamMajer Please review.
- Modify the `Password` field in `CreateUserOption` struct to remove the
`Required` tag
- Update the `v1_json.tmpl` template to include the `email` field and
remove the `password` field
---------
Signed-off-by: Bo-Yi Wu <appleboy.tw@gmail.com>
Update golang.org/x/crypto for CVE-2023-48795 and update other packages.
`go-git` is not updated because it needs time to figure out why some
tests fail.
- If a topic has zero repository count, it means that none of the
repositories are using that topic, that would make them 'useless' to
keep. One caveat is that if that topic is going to be used in the
future, it will be added again to the database, but simply with a new
ID.
Refs: https://codeberg.org/forgejo/forgejo/pulls/1964
Co-authored-by: Gusted <postmaster@gusted.xyz>
- Remove `ObjectFormatID`
- Remove function `ObjectFormatFromID`.
- Use `Sha1ObjectFormat` directly but not a pointer because it's an
empty struct.
- Store `ObjectFormatName` in `repository` struct
- When a repository is orphaned and has objects stored in any of the
storages such as repository avatar or attachments the delete function
would error, because the storage module wasn't initalized.
- Add code to initialize the storage module.
Refs: https://codeberg.org/forgejo/forgejo/pulls/1954
Co-authored-by: Gusted <postmaster@gusted.xyz>
Refactor Hash interfaces and centralize hash function. This will allow
easier introduction of different hash function later on.
This forms the "no-op" part of the SHA256 enablement patch.
## Changes
- Add deprecation warning to `Token` and `AccessToken` authentication
methods in swagger.
- Add deprecation warning header to API response. Example:
```
HTTP/1.1 200 OK
...
Warning: token and access_token API authentication is deprecated
...
```
- Add setting `DISABLE_QUERY_AUTH_TOKEN` to reject query string auth
tokens entirely. Default is `false`
## Next steps
- `DISABLE_QUERY_AUTH_TOKEN` should be true in a subsequent release and
the methods should be removed in swagger
- `DISABLE_QUERY_AUTH_TOKEN` should be removed and the implementation of
the auth methods in question should be removed
## Open questions
- Should there be further changes to the swagger documentation?
Deprecation is not yet supported for security definitions (coming in
[OpenAPI Spec version
3.2.0](https://github.com/OAI/OpenAPI-Specification/issues/2506))
- Should the API router logger sanitize urls that use `token` or
`access_token`? (This is obviously an insufficient solution on its own)
---------
Co-authored-by: delvh <dev.lh@web.de>
1. Do not sort the "checks" slice again and again when "Register", it
just wastes CPU when the Gitea instance runs
2. If a check doesn't exist, tell the end user
3. Add some tests
The function `GetByBean` has an obvious defect that when the fields are
empty values, it will be ignored. Then users will get a wrong result
which is possibly used to make a security problem.
To avoid the possibility, this PR removed function `GetByBean` and all
references.
And some new generic functions have been introduced to be used.
The recommand usage like below.
```go
// if query an object according id
obj, err := db.GetByID[Object](ctx, id)
// query with other conditions
obj, err := db.Get[Object](ctx, builder.Eq{"a": a, "b":b})
```
It will fix#28268 .
<img width="1313" alt="image"
src="https://github.com/go-gitea/gitea/assets/9418365/cb1e07d5-7a12-4691-a054-8278ba255bfc">
<img width="1318" alt="image"
src="https://github.com/go-gitea/gitea/assets/9418365/4fd60820-97f1-4c2c-a233-d3671a5039e9">
## ⚠️ BREAKING ⚠️
But need to give up some features:
<img width="1312" alt="image"
src="https://github.com/go-gitea/gitea/assets/9418365/281c0d51-0e7d-473f-bbed-216e2f645610">
However, such abandonment may fix#28055 .
## Backgroud
When the user switches the dashboard context to an org, it means they
want to search issues in the repos that belong to the org. However, when
they switch to themselves, it means all repos they can access because
they may have created an issue in a public repo that they don't own.
<img width="286" alt="image"
src="https://github.com/go-gitea/gitea/assets/9418365/182dcd5b-1c20-4725-93af-96e8dfae5b97">
It's a confusing design. Think about this: What does "In your
repositories" mean when the user switches to an org? Repos belong to the
user or the org?
Whatever, it has been broken by #26012 and its following PRs. After the
PR, it searches for issues in repos that the dashboard context user owns
or has been explicitly granted access to, so it causes #28268.
## How to fix it
It's not really difficult to fix it. Just extend the repo scope to
search issues when the dashboard context user is the doer. Since the
user may create issues or be mentioned in any public repo, we can just
set `AllPublic` to true, which is already supported by indexers. The DB
condition will also support it in this PR.
But the real difficulty is how to count the search results grouped by
repos. It's something like "search issues with this keyword and those
filters, and return the total number and the top results. **Then, group
all of them by repo and return the counts of each group.**"
<img width="314" alt="image"
src="https://github.com/go-gitea/gitea/assets/9418365/5206eb20-f8f5-49b9-b45a-1be2fcf679f4">
Before #26012, it was being done in the DB, but it caused the results to
be incomplete (see the description of #26012).
And to keep this, #26012 implement it in an inefficient way, just count
the issues by repo one by one, so it cannot work when `AllPublic` is
true because it's almost impossible to do this for all public repos.
1bfcdeef4c/modules/indexer/issues/indexer.go (L318-L338)
## Give up unnecessary features
We may can resovle `TODO: use "group by" of the indexer engines to
implement it`, I'm sure it can be done with Elasticsearch, but IIRC,
Bleve and Meilisearch don't support "group by".
And the real question is, does it worth it? Why should we need to know
the counts grouped by repos?
Let me show you my search dashboard on gitea.com.
<img width="1304" alt="image"
src="https://github.com/go-gitea/gitea/assets/9418365/2bca2d46-6c71-4de1-94cb-0c9af27c62ff">
I never think the long repo list helps anything.
And if we agree to abandon it, things will be much easier. That is this
PR.
## TODO
I know it's important to filter by repos when searching issues. However,
it shouldn't be the way we have it now. It could be implemented like
this.
<img width="1316" alt="image"
src="https://github.com/go-gitea/gitea/assets/9418365/99ee5f21-cbb5-4dfe-914d-cb796cb79fbe">
The indexers support it well now, but it requires some frontend work,
which I'm not good at. So, I think someone could help do that in another
PR and merge this one to fix the bug first.
Or please block this PR and help to complete it.
Finally, "Switch dashboard context" is also a design that needs
improvement. In my opinion, it can be accomplished by adding filtering
conditions instead of "switching".
The summary string ends up in the database, and (at least) MySQL &
PostgreSQL require valid UTF8 strings.
Fixes#28178
Co-authored-by: Darrin Smart <darrin@filmlight.ltd.uk>
Previously only the first term had to be matched. That default
Meilisearch behavior makes sense for e.g. some kind of autocomplete to
find and select a single result. But for filtering issues it means you
can't narrow down results by adding more terms.
This is also more consistent with other indexers and GitHub.
---
Reference:
https://www.meilisearch.com/docs/reference/api/search#matching-strategy
The git command may operate the git directory (add/remove) files in any
time.
So when the code iterates the directory, some files may disappear during
the "walk". All "IsNotExist" errors should be ignored.
Fix#26765
gitea doctor failed at checking and fixing 'delete-orphaned-repos',
because table name 'user' needs quoting to be correctly recognized by at
least PostgreSQL.
fixes#28199
- Currently the repository description uses the same sanitizer as a
normal markdown document. This means that element such as heading and
images are allowed and can be abused.
- Create a minimal restricted sanitizer for the repository description,
which only allows what the postprocessor currently allows, which are
links and emojis.
- Added unit testing.
- Resolves https://codeberg.org/forgejo/forgejo/issues/1202
- Resolves https://codeberg.org/Codeberg/Community/issues/1122
(cherry picked from commit 631c87cc2347f0036a75dcd21e24429bbca28207)
Co-authored-by: Gusted <postmaster@gusted.xyz>
Fix#25473
Although there was `m.Post("/login/oauth/access_token", CorsHandler()...`,
it never really worked, because it still lacks the "OPTIONS" handler.
Fixes#27819
We have support for two factor logins with the normal web login and with
basic auth. For basic auth the two factor check was implemented at three
different places and you need to know that this check is necessary. This
PR moves the check into the basic auth itself.
- On user deletion, delete action runners that the user has created.
- Add a database consistency check to remove action runners that have
nonexistent belonging owner.
- Resolves https://codeberg.org/forgejo/forgejo/issues/1720
(cherry picked from commit 009ca7223dab054f7f760b7ccae69e745eebfabb)
Co-authored-by: Gusted <postmaster@gusted.xyz>
This patchset changes the connection string builder to use net.URL and
the host/port parser to use the stdlib function for splitting host from
port. It also adds a footnote about a potentially required portnumber
for postgres UNIX sockets.
Fixes: #24552
This PR adds a prefix path for all minio storage and override base path
will override the path.
The previous behavior is undefined officially, so it will be marked as
breaking.
After many refactoring PRs for the "locale" and "template context
function", now the ".locale" is not needed for web templates any more.
This PR does a clean up for:
1. Remove `ctx.Data["locale"]` for web context.
2. Use `ctx.Locale` in `500.tmpl`, for consistency.
3. Add a test check for `500 page` locale usage.
4. Remove the `Str2html` and `DotEscape` from mail template context
data, they are copy&paste errors introduced by #19169 and #16200 . These
functions are template functions (provided by the common renderer), but
not template data variables.
5. Make email `SendAsync` function mockable (I was planning to add more
tests but it would make this PR much too complex, so the tests could be
done in another PR)
See https://github.com/go-gitea/gitea/pull/27718#issuecomment-1773743014
. Add a test to ensure its behavior.
Why this test uses `ProjectBoardID=0`? Because in `SearchOptions`,
`ProjectBoardID=0` means what it is. But in `IssueOptions`,
`ProjectBoardID=0` means there is no condition, and
`ProjectBoardID=db.NoConditionID` means the board ID = 0.
It's really confusing. Probably it's better to separate the db search
engine and the other issue search code. It's really two different
systems. As far as I can see, `IssueOptions` is not necessary for most
of the code, which has very simple issue search conditions.
- The label HTML contained a quote that wasn't being closed.
Refs: https://codeberg.org/forgejo/forgejo/pulls/1651
(cherry picked from commit e2bc2c9a1fff482c49dbeb3a51e4e1c698bf506c)
Co-authored-by: Gusted <postmaster@gusted.xyz>
1. remove unused function `MoveIssueAcrossProjectBoards`
2. extract the project board condition into a function
3. use db.NoCondition instead of -1. (BTW, the usage of db.NoCondition
is too confusing. Is there any way to avoid that?)
4. remove the unnecessary comment since the ctx refactor is completed.
5. Change `b.ID != 0` to `b.ID > 0`. It's more intuitive but I think
they're the same since board ID can't be negative.
When `webhook.PROXY_URL` has been set, the old code will check if the
proxy host is in `ALLOWED_HOST_LIST` or reject requests through the
proxy. It requires users to add the proxy host to `ALLOWED_HOST_LIST`.
However, it actually allows all requests to any port on the host, when
the proxy host is probably an internal address.
But things may be even worse. `ALLOWED_HOST_LIST` doesn't really work
when requests are sent to the allowed proxy, and the proxy could forward
them to any hosts.
This PR fixes it by:
- If the proxy has been set, always allow connectioins to the host and
port.
- Check `ALLOWED_HOST_LIST` before forwarding.
Closes#27455
> The mechanism responsible for long-term authentication (the 'remember
me' cookie) uses a weak construction technique. It will hash the user's
hashed password and the rands value; it will then call the secure cookie
code, which will encrypt the user's name with the computed hash. If one
were able to dump the database, they could extract those two values to
rebuild that cookie and impersonate a user. That vulnerability exists
from the date the dump was obtained until a user changed their password.
>
> To fix this security issue, the cookie could be created and verified
using a different technique such as the one explained at
https://paragonie.com/blog/2015/04/secure-authentication-php-with-long-term-persistence#secure-remember-me-cookies.
The PR removes the now obsolete setting `COOKIE_USERNAME`.
assert.Fail() will continue to execute the code while assert.FailNow()
not. I thought those uses of assert.Fail() should exit immediately.
PS: perhaps it's a good idea to use
[require](https://pkg.go.dev/github.com/stretchr/testify/require)
somewhere because the assert package's default behavior does not exit
when an error occurs, which makes it difficult to find the root error
reason.
Fix #27541
The INI package has a quirk: by default, the keys are inherited.
When maintaining the keys, the newly added sub key should not be
affected by the parent key.
Part of https://github.com/go-gitea/gitea/issues/27097:
- `gitea` theme is renamed to `gitea-light`
- `arc-green` theme is renamed to `gitea-dark`
- `auto` theme is renamed to `gitea-auto`
I put both themes in separate CSS files, removing all colors from the
base CSS. Existing users will be migrated to the new theme names. The
dark theme recolor will follow in a separate PR.
## ⚠️ BREAKING ⚠️
1. If there are existing custom themes with the names `gitea-light` or
`gitea-dark`, rename them before this upgrade and update the `theme`
column in the `user` table for each affected user.
2. The theme in `<html>` has moved from `class="theme-name"` to
`data-theme="name"`, existing customizations that depend on should be
updated.
---------
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
This pull request is a minor code cleanup.
From the Go specification (https://go.dev/ref/spec#For_range):
> "1. For a nil slice, the number of iterations is 0."
> "3. If the map is nil, the number of iterations is 0."
`len` returns 0 if the slice or map is nil
(https://pkg.go.dev/builtin#len). Therefore, checking `len(v) > 0`
before a loop is unnecessary.
---
At the time of writing this pull request, there wasn't a lint rule that
catches these issues. The closest I could find is
https://staticcheck.dev/docs/checks/#S103
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
This PR reduces the complexity of the system setting system.
It only needs one line to introduce a new option, and the option can be
used anywhere out-of-box.
It is still high-performant (and more performant) because the config
values are cached in the config system.
With this PR we added the possibility to configure the Actions timeouts
values for killing tasks/jobs.
Particularly this enhancement is closely related to the `act_runner`
configuration reported below:
```
# The timeout for a job to be finished.
# Please note that the Gitea instance also has a timeout (3h by default) for the job.
# So the job could be stopped by the Gitea instance if it's timeout is shorter than this.
timeout: 3h
```
---
Setting the corresponding key in the INI configuration file, it is
possible to let jobs run for more than 3 hours.
Signed-off-by: Francesco Antognazza <francesco.antognazza@gmail.com>
Part of #27065
This PR touches functions used in templates. As templates are not static
typed, errors are harder to find, but I hope I catch it all. I think
some tests from other persons do not hurt.
Follow #23465 and #25624
This PR introduces the following improvements:
- We do not need to call `GetTags` to get tags because tags have been
loaded by `RepoAssignment`
ef90fdbd1d/modules/context/repo.go (L663-L668)
- Similarly, the number of tags and releases also have been loaded by
`RepoAssignment`, so the related code has been removed from the
handlers. The query condition of `GetReleaseCountByRepoID` in
`RepoAssignment` has been changed to include draft releases.
ef90fdbd1d/modules/context/repo.go (L538-L551)
- `releasesOrTags` function has been removed. The code for rendering
releases list and tags list moved to `Releases` and `TagList`
respectively.
- Modify the deleted branch orphan check to check for the new table
instead.
- Regression from 6e19484f4d
- Resolves https://codeberg.org/forgejo/forgejo/issues/1522
(cherry picked from commit c1d888686fe445e4edecb9d835c5b3893b574b75)
Co-authored-by: Gusted <postmaster@gusted.xyz>
This PR removed `unittest.MainTest` the second parameter
`TestOptions.GiteaRoot`. Now it detects the root directory by current
working directory.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
I noticed, that the push mirrors endpoint, is the only endpoint which
returns the times in long format rather than as time.Time().
I think the behavior should be consistent across the project.
----
## ⚠️ BREAKING ⚠️
This PR changes the time format used in API responses for all
push_mirror endpoints which return a push mirror.
---------
Co-authored-by: Giteabot <teabot@gitea.io>
- Refactor lfs request code
- The original code uses `performRequest` function to create the
request, uses a callback to modify the request, and then send the
request.
- Now it's replaced with `createRequest` that only creates request and
`performRequest` that only sends the request.
- Reuse `createRequest` and `performRequest` in `http_client.go` and
`transferadapter.go`
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Closes#26329
This PR adds the ability to ignore revisions specified in the
`.git-blame-ignore-revs` file in the root of the repository.
The banner is displayed in this case. I intentionally did not add a UI
way to bypass the ignore file (same behaviour as Github) but you can add
`?bypass-blame-ignore=true` to the url manually.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
This PR adds a new field `RemoteAddress` to both mirror types which
contains the sanitized remote address for easier (database) access to
that information. Will be used in the audit PR if merged.
As described in the title.
Some points:
1. Why need those tests?
Because `buildIssueOverview` is not well tested, there are several
continuous bugs in the issue overview webpage.
2. Why in indexer_test.go?
It's hard to put those tests in `./modules/indexer/issue/db/db_test.go`
because those tests need 'real' data in db mocked by fixtures instead of
random data in `./modules/indexer/issue/internal/tests`. When using
'real' data(`unittest.PrepareTestDatabase`), `InitIssueIndexer` and the
package `init()` function of `indexer` are required to init indexer.
3. Why only db?
The other three indexer engines are well tested by random data and it's
okay to also test them with 'real' data in db mocked by fixtures. Any
follow-up PR is welcome.
4. Those tests are really basic, any more complicated tests are welcome.
5. I think it's also necessary to add tests in `TestAPISearchIssues`
in`api_test_issue.go` and `TestIssues` in `home_test.go`
Part of #27065
This reduces the usage of `db.DefaultContext`. I think I've got enough
files for the first PR. When this is merged, I will continue working on
this.
Considering how many files this PR affect, I hope it won't take to long
to merge, so I don't end up in the merge conflict hell.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Since the issue indexer has been refactored, the issue overview webpage
is built by the `buildIssueOverview` function and underlying
`indexer.Search` function and `GetIssueStats` instead of
`GetUserIssueStats`. So the function is no longer used.
I moved the relevant tests to `indexer_test.go` and since the search
option changed from `IssueOptions` to `SearchOptions`, most of the tests
are useless now.
We need more tests about the db indexer because those tests are highly
connected with the issue overview webpage and now this page has several
bugs.
Any advice about those test cases is appreciated.
---------
Co-authored-by: CaiCandong <50507092+CaiCandong@users.noreply.github.com>
- In org mode you can specify an description for media via the following
syntax `[[description][media link]]`. The description is then used as
title or alt.
- This patch fixes the rendering of the description by seperating the
description and non-description cases and using `org.String()`.
- Added unit tests.
- Inspired by
6eb20dbda9/org/html_writer.go (L406-L427)
- Resolves https://codeberg.org/Codeberg/Community/issues/848
(cherry picked from commit 8b8aab83113b34bade61964e2097ed497abc39e9)
Co-authored-by: Gusted <postmaster@gusted.xyz>
Fix#26723
Add `ChangeDefaultBranch` to the `notifier` interface and implement it
in `indexerNotifier`. So when changing the default branch,
`indexerNotifier` sends a message to the `indexer queue` to update the
index.
---------
Co-authored-by: techknowlogick <matti@mdranta.net>
Should BucketExists (HeadBucket) fail because of an error related to
the connection rather than the existence of the bucket, no information
is available and the admin is left guessing.
https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadBucket.html
> This action is useful to determine if a bucket exists and you have
> permission to access it. The action returns a 200 OK if the bucket
> exists and you have permission to access it.
>
> If the bucket does not exist or you do not have permission to access
> it, the HEAD request returns a generic 400 Bad Request, 403
> Forbidden or 404 Not Found code. A message body is not included, so
> you cannot determine the exception beyond these error codes.
GetBucketVersioning is used instead and exclusively dedicated to
asserting if using the connection does not return a BadRequest.
If it does the NewMinioStorage logs an error and returns. Otherwise
it keeps going knowing that BucketExists is not going to fail for
reasons unrelated to the existence of the bucket and the permissions
to access it.
(cherry picked from commit d1df4b3bc62e5e61893a923f1c4b58f084eb03af)
Refs: https://codeberg.org/forgejo/forgejo/issues/1338
Unfortunately, when a system setting hasn't been stored in the database,
it cannot be cached.
Meanwhile, this PR also uses context cache for push email avatar display
which should avoid to read user table via email address again and again.
According to my local test, this should reduce dashboard elapsed time
from 150ms -> 80ms .
If the AppURL(ROOT_URL) is an HTTPS URL, then the COOKIE_SECURE's
default value should be true.
And, if a user visits an "http" site with "https" AppURL, they won't be
able to login, and they should have been warned. The only problem is
that the "language" can't be set either in such case, while I think it
is not a serious problem, and it could be fixed easily if needed.
A set of terminology, along with a broader description, can help more
people engage with the Gitea queue system, providing insights and
ensuring its correct use.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
This feature was removed by #22219 to avoid possible CSRF attack.
This PR takes reverseproxy auth for API back but with default disabled.
To prevent possbile CSRF attack, the responsibility will be the
reverseproxy but not Gitea itself.
For those want to enable this `ENABLE_REVERSE_PROXY_AUTHENTICATION_API`,
they should know what they are doing.
---------
Co-authored-by: Giteabot <teabot@gitea.io>
Currently, Artifact does not have an expiration and automatic cleanup
mechanism, and this feature needs to be added. It contains the following
key points:
- [x] add global artifact retention days option in config file. Default
value is 90 days.
- [x] add cron task to clean up expired artifacts. It should run once a
day.
- [x] support custom retention period from `retention-days: 5` in
`upload-artifact@v3`.
- [x] artifacts link in actions view should be non-clickable text when
expired.
1. The `og:description` should be "a one to two sentence description of
your object"
* It shouldn't output all the user inputted content -- it would be
pretty huge.
* Maybe it only needs at most 300 bytes.
2. Do not render commit message as HTML
- While doing some sanity checks over OpenSSH's code for how they handle
certificates authentication. I stumbled on an condition that checks the
certificate type is really an user certificate on the server-side
authentication. This checks seems to be a formality and just for the
sake of good domain seperation, because an user and host certificate
don't differ in their generation, verification or flags that can be
included.
- Add this check to the builtin SSH server to stay close to the
unwritten SSH specification.
- This is an breaking change for setups where the builtin SSH server is
being used and for some reason host certificates were being used for
authentication.
-
(cherry picked from commit de35b141b79a3d6efe2127ed2c73fd481515e481)
Refs: https://codeberg.org/forgejo/forgejo/pulls/1172
## ⚠️ BREAKING ⚠️
Like OpenSSH, the built-in SSH server will now only accept SSH user
certificates, not server certificates.
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-authored-by: Giteabot <teabot@gitea.io>
1. The old `prepareQueryArg` did double-unescaping of form value.
2. By the way, remove the unnecessary `ctx.Flash = ...` in
`MockContext`.
Co-authored-by: Giteabot <teabot@gitea.io>
Just like `models/unittest`, the testing helper functions should be in a
separate package: `contexttest`
And complete the TODO:
> // TODO: move this function to other packages, because it depends on
"models" package
Backtick syntax now works in repo description too. Also, I replaced the
CSS for this was a new single class, making it more flexible and not
dependent on a parent. Also, very slightly reduced font size from 16.8px
to 16px.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
From the Go specification:
> "1. For a nil slice, the number of iterations is 0."
https://go.dev/ref/spec#For_range
Therefore, an additional nil check for before the loop is unnecessary.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Related to: #8312#26491
In migration v109, we only added a new column `CanCreateOrgRepo` in Team
table, but not initial the value of it.
This may cause bug like #26491.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
According to the GitHub API Spec:
https://docs.github.com/en/rest/actions/secrets?apiVersion=2022-11-28#create-or-update-an-organization-secret
Merge the Create and Update secret into a single API.
- Remove the `CreateSecretOption` struct and replace it with
`CreateOrUpdateSecretOption` in `modules/structs/secret.go`
- Update the `CreateOrUpdateOrgSecret` function in
`routers/api/v1/org/action.go` to use `CreateOrUpdateSecretOption`
instead of `UpdateSecretOption`
- Remove the `CreateOrgSecret` function in
`routers/api/v1/org/action.go` and replace it with
`CreateOrUpdateOrgSecret`
- Update the Swagger documentation in
`routers/api/v1/swagger/options.go` and `templates/swagger/v1_json.tmpl`
to reflect the changes in the struct names and function names
Signed-off-by: Bo-Yi Wu <appleboy.tw@gmail.com>
The web context (modules/context.Context) is quite complex, it's
difficult for the callers to initialize correctly.
This PR introduces a `NewWebContext` function, to make sure the web
context have the same behavior for different cases.
- Resolves https://codeberg.org/forgejo/forgejo/issues/580
- Return a `upload_field` to any release API response, which points to
the API URL for uploading new assets.
- Adds unit test.
- Adds integration testing to verify URL is returned correctly and that
upload endpoint actually works
---------
Co-authored-by: Gusted <postmaster@gusted.xyz>
Hi,
We'd like to add merge files files to GetCommitFileStatus fucntions so
API returns the list of all the files associated to a merged pull
request commit, like GitHub API does.
The list of affectedFiles for an API commit is fetched from toCommit()
function in routers/api/v1/repo/commits.go, and API was returning no
file in case of a pull request with no conflict, or just files
associated to the confict resolution, but NOT the full list of merged
files.
This would lead to situations where a CI polling a repo for changes
could miss some file changes due to API returning an empty / partial
list in case of such merged pull requests. (Hope this makes sense :) )
NOTE: I'd like to add a unittest in
integrations/api_repo_git_commits_test.go but failed to understand how
to add my own test bare repo so I can make a test on a merged pull
request commit to check for affectedFiles.
Is there a merged pull request in there that I could use maybe?
Could someone please direct me to the relevant ressources with
informations on how to do that please?
Thanks for your time,
Laurent.
---------
Co-authored-by: Thomas Desveaux <desveaux.thomas@gmail.com>
Replace #22751
1. only support the default branch in the repository setting.
2. autoload schedule data from the schedule table after starting the
service.
3. support specific syntax like `@yearly`, `@monthly`, `@weekly`,
`@daily`, `@hourly`
## How to use
See the [GitHub Actions
document](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#schedule)
for getting more detailed information.
```yaml
on:
schedule:
- cron: '30 5 * * 1,3'
- cron: '30 5 * * 2,4'
jobs:
test_schedule:
runs-on: ubuntu-latest
steps:
- name: Not on Monday or Wednesday
if: github.event.schedule != '30 5 * * 1,3'
run: echo "This step will be skipped on Monday and Wednesday"
- name: Every time
run: echo "This step will always run"
```
Signed-off-by: Bo-Yi.Wu <appleboy.tw@gmail.com>
---------
Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
This PR has multiple parts, and I didn't split them because
it's not easy to test them separately since they are all about the
dashboard page for issues.
1. Support counting issues via indexer to fix#26361
2. Fix repo selection so it also fixes#26653
3. Keep keywords in filter links.
The first two are regressions of #26012.
After:
https://github.com/go-gitea/gitea/assets/9418365/71dfea7e-d9e2-42b6-851a-cc081435c946
Thanks to @CaiCandong for helping with some tests.
- Add a new `CreateSecretOption` struct for creating secrets
- Implement a `CreateOrgSecret` function to create a secret in an
organization
- Add a new route in `api.go` to handle the creation of organization
secrets
- Update the Swagger template to include the new `CreateOrgSecret` API
endpoint
---------
Signed-off-by: appleboy <appleboy.tw@gmail.com>
Previously, `err` was defined above, checked for `err == nil` and used
nowhere else.
Hence, the result of `convertMinioErr` would always be `nil`.
This leads to a NPE further down the line.
That is not intentional, it should convert the error of the most recent
operation, not one of its predecessors.
Found through
https://discord.com/channels/322538954119184384/322538954119184384/1143185780206993550.
- Added new tests to cover corner cases
- Replace existing regex with new one
Closes#26551
---
As @silverwind suggested, I started from
[validate-npm-package-name](https://github.com/npm/validate-npm-package-name),
but found this solution too complicated.
Then I tried to fix existing regex myself, but thought, that exclude all
restricted symbols is harder, than set only allowed symbols.
Then I search a bit more and found
[package-name-regex](https://github.com/dword-design/package-name-regex)
and regex from it works for all new test cases.
Let me know, if more information or help with this PR is needed.
Fix#26536
Follow #26012
Whatever the comment type is, always update the issue indexer. So the
issue indexer will be updated when there is a change in Status,
Assignee, Label, and so on.
I added the logic for `NotifyUpdateComment`, but missed it for
`NotifyCreateIssueComment` and `NotifyDeleteComment`.
- Add a new function `CountOrgSecrets` in the file
`models/secret/secret.go`
- Add a new file `modules/structs/secret.go`
- Add a new function `ListActionsSecrets` in the file
`routers/api/v1/api.go`
- Add a new file `routers/api/v1/org/action.go`
- Add a new function `listActionsSecrets` in the file
`routers/api/v1/org/action.go`
go-sdk: https://gitea.com/gitea/go-sdk/pulls/629
---------
Signed-off-by: Bo-Yi Wu <appleboy.tw@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <matti@mdranta.net>
Co-authored-by: Giteabot <teabot@gitea.io>
"ogg" is just a "container" format for audio and video.
Golang's `DetectContentType` only reports "application/ogg" for
potential ogg files.
Actually it could do more "guess" to see whether it is a audio file or a
video file.
## Archived labels
This adds the structure to allow for archived labels.
Archived labels are, just like closed milestones or projects, a medium to hide information without deleting it.
It is especially useful if there are outdated labels that should no longer be used without deleting the label entirely.
## Changes
1. UI and API have been equipped with the support to mark a label as archived
2. The time when a label has been archived will be stored in the DB
## Outsourced for the future
There's no special handling for archived labels at the moment.
This will be done in the future.
## Screenshots
Part of https://github.com/go-gitea/gitea/issues/25237
---------
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
This PR rewrites the function `getStorage` and make it more clear.
Include tests from #26435, thanks @earl-warren
---------
Co-authored-by: Earl Warren <contact@earl-warren.org>
Close stdout correctly for "git blame", otherwise the failed "git blame"
would case the request hanging forever.
And "os.Stderr" should never (seldom) be used as git command's stderr
When users put the secrets into a file (GITEA__sec__KEY__FILE), the
newline sometimes is different to avoid (eg: echo/vim/...)
So the last newline could be removed when reading, it makes the users
easier to maintain the secret files.
Co-authored-by: Giteabot <teabot@gitea.io>
For some reason, the permission of the client_id and secret may cannot
create bucket, so now we will check whether bucket does exist first and
then try to create a bucket if it doesn't exist.
Try to fix#25984
Co-authored-by: silverwind <me@silverwind.io>
In the `RepoRefForAPI()` context function `CommitID` is not set if `ref`
is used. It is set correctly for other if/else branches where `Commit`
is set. It doesn't appear that any routes that use `RepoRefForAPI()`
also use `CommitID` but that may be the case in the future.
## Changes
- Sets `ctx.Repo.CommitID` when `ref` is explicitly used for api routes
that use `RepoRefForAPI()`
The MinIO client isn't redirecting to the correct AWS endpoint if a
non-default data center is used.
In my use case I created an AWS bucket at `eu-central-1` region. Because
of the missing region initialization of the client the default
`us-east-1` API endpoint is used returning a `301 Moved Permanently`
response that's not handled properly by MinIO client. This in return
aborts using S3 storage on AWS as the `BucketExists()` call will fail
with the http moved error.
MinIO client trace shows the issue:
```text
---------START-HTTP---------
HEAD / HTTP/1.1
Host: xxxxxxxxxxx-prod-gitea-data.s3.dualstack.us-east-1.amazonaws.com
User-Agent: MinIO (windows; amd64) minio-go/v7.0.61
Authorization: AWS4-HMAC-SHA256 Credential=**REDACTED**/20230809/accesspoint.eu-central-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=**REDACTED**
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20230809T141143Z
HTTP/1.1 301 Moved Permanently
Connection: close
Content-Type: application/xml
Date: Wed, 09 Aug 2023 14:11:43 GMT
Server: AmazonS3
X-Amz-Bucket-Region: eu-central-1
X-Amz-Id-2: UK7wfeYi0HcTcytNvQ3wTAZ5ZP1mOSMnvRZ9Fz4xXzeNsS47NB/KfFx2unFxo3L7XckHpMNPPVo=
X-Amz-Request-Id: S1V2MJV8SZ11GEVN
---------END-HTTP---------
```
Co-authored-by: Heiko Besemann <heiko.besemann@qbeyond.de>
This PR is an extended implementation of #25189 and builds upon the
proposal by @hickford in #25653, utilizing some ideas proposed
internally by @wxiaoguang.
Mainly, this PR consists of a mechanism to pre-register OAuth2
applications on startup, which can be enabled or disabled by modifying
the `[oauth2].DEFAULT_APPLICATIONS` parameter in app.ini. The OAuth2
applications registered this way are being marked as "locked" and
neither be deleted nor edited over UI to prevent confusing/unexpected
behavior. Instead, they're being removed if no longer enabled in config.
The implemented mechanism can also be used to pre-register other OAuth2
applications in the future, if wanted.
Co-authored-by: hickford <mirth.hickford@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
---------
Co-authored-by: M Hickford <mirth.hickford@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
There are 2 kinds of ".Editorconfig" in code, one is `JSON string` for
the web edtior, another is `*editorconfig.Editorconfig` for the file
rendering (used by `TabSizeClass`)
This PR distinguish them with different names.
And by the way, change the default tab size from 8 to 4, I think few
people would like to use 8-size tabs nowadays.
Before:
* `{{.locale.Tr ...}}`
* `{{$.locale.Tr ...}}`
* `{{$.root.locale.Tr ...}}`
* `{{template "sub" .}}`
* `{{template "sub" (dict "locale" $.locale)}}`
* `{{template "sub" (dict "root" $)}}`
* .....
With context function: only need to `{{ctx.Locale.Tr ...}}`
The "ctx" could be considered as a super-global variable for all
templates including sub-templates.
To avoid potential risks (any bug in the template context function
package), this PR only starts using "ctx" in "head.tmpl" and
"footer.tmpl" and it has a "DataRaceCheck". If there is anything wrong,
the code can be fixed or reverted easily.
- Currently the post processing will transform all issue indexes (such as `#6`) into a clickable link.
- This makes sense in an situation like issues or PRs,
where referencing to other issues is quite common
and only referencing their issue index is an handy and efficient way to do it.
- Currently this is also run for documents
(which is the user profile and viewing rendered files),
but in those situations it's less common to reference issues by their index and instead could mean something else.
- This patch disables this post processing for issue index for documents. Matches Github's behavior.
- Added unit tests.
- Resolves https://codeberg.org/Codeberg/Community/issues/1120
Co-authored-by: Gusted <postmaster@gusted.xyz>
Follow #25229
Copy from
https://github.com/go-gitea/gitea/pull/26290#issuecomment-1663135186
The bug is that we cannot get changed files for the
`pull_request_target` event. This event runs in the context of the base
branch, so we won't get any changes if we call
`GetFilesChangedSinceCommit` with `PullRequest.Base.Ref`.
Fix#26064
Some git commands should use parent context, otherwise it would exit too
early (by the default timeout, 10m), and the "cmd.Wait" waits till the
pipes are closed.
This PR will fix#26264, caused by #23911.
The package configuration derive is totally wrong when storage type is
local in that PR.
This PR fixed the inherit logic when storage type is local with some
unit tests.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Fixes#26270.
Co-Author: @wxiaoguang
Thanks @lunny for providing this solution
As
https://github.com/go-gitea/gitea/issues/26270#issuecomment-1661695151
said, at present we cannot get the names of changed files correctly when
the `OldCommitID` is `EmptySHA`. In this PR, the `GetCommitFilesChanged`
method is added and will be used to get the changed files by commit ID.
References:
- https://stackoverflow.com/a/424142
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
1. Fix the wrong document (add the missing `MODE=`)
2. Add a more friendly log message to tell users to add `MODE=` in their
config
Co-authored-by: Giteabot <teabot@gitea.io>
Not too important, but I think that it'd be a pretty neat touch.
Also fixes some layout bugs introduced by a previous PR.
---------
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-authored-by: Caesar Schinas <caesar@caesarschinas.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Fix#24662.
Replace #24822 and #25708 (although it has been merged)
## Background
In the past, Gitea supported issue searching with a keyword and
conditions in a less efficient way. It worked by searching for issues
with the keyword and obtaining limited IDs (as it is heavy to get all)
on the indexer (bleve/elasticsearch/meilisearch), and then querying with
conditions on the database to find a subset of the found IDs. This is
why the results could be incomplete.
To solve this issue, we need to store all fields that could be used as
conditions in the indexer and support both keyword and additional
conditions when searching with the indexer.
## Major changes
- Redefine `IndexerData` to include all fields that could be used as
filter conditions.
- Refactor `Search(ctx context.Context, kw string, repoIDs []int64,
limit, start int, state string)` to `Search(ctx context.Context, options
*SearchOptions)`, so it supports more conditions now.
- Change the data type stored in `issueIndexerQueue`. Use
`IndexerMetadata` instead of `IndexerData` in case the data has been
updated while it is in the queue. This also reduces the storage size of
the queue.
- Enhance searching with Bleve/Elasticsearch/Meilisearch, make them
fully support `SearchOptions`. Also, update the data versions.
- Keep most logic of database indexer, but remove
`issues.SearchIssueIDsByKeyword` in `models` to avoid confusion where is
the entry point to search issues.
- Start a Meilisearch instance to test it in unit tests.
- Add unit tests with almost full coverage to test
Bleve/Elasticsearch/Meilisearch indexer.
---------
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
- Configure `setting.CacheService.TTL` which will force the code to go
trough the caching mechanism.
- Remove the TODO and uncomment the test code.
(cherry picked from commit a201f2f1896c1a086f3e37f86b1306d44c07f2e4)
Refs: https://codeberg.org/forgejo/forgejo/pulls/974
---------
Co-authored-by: Gusted <postmaster@gusted.xyz>
- The `NoBetterThan` function can only handle comparisons between
"pending," "success," "error," and "failure." For any other comparison,
we directly return false. This prevents logic errors like the one in
#26121.
- The callers of the `NoBetterThan` function should also avoid making
incomparable calls.
---------
Co-authored-by: yp05327 <576951401@qq.com>
Co-authored-by: puni9869 <80308335+puni9869@users.noreply.github.com>
The JSONRedirect/JSONOK/JSONError functions were put into "Base" context
incorrectly, it would cause abuse.
Actually, they are for "web context" only, so, move them to the correct
place.
And by the way, use them to simplify old code: +75 -196
This PR includes #26007 's changes but have a UI to prompt administrator
about the deprecated settings as well as the log or console warning.
Then users will have enough time to notice the problem and don't have
surprise like before.
<img width="1293" alt="图片"
src="https://github.com/go-gitea/gitea/assets/81045/c33355f0-1ea7-4fb3-ad43-cd23cd15391d">
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
This is a simple PR which moves the `GetListener` function to a
`DefaultGetListener` function, and changes `GetListener` to be a
variable which by default points to the `DefaultGetListener` function.
This allows people who may exist quasi-downstream of Gitea to create
alternate "GetListener" functions, with identical signatures, which
return different implementations of the `net.Listener` interface. This
approach is expressly intended to be non-invasive and have the least
possible impact on the gitea codebase. A previous version of this idea
was rejected before: https://github.com/go-gitea/gitea/issues/15544 but
because of issues like: https://github.com/go-gitea/gitea/issues/22335 I
**really** think that recommending people configure proxies by hand is
exactly the wrong way to do things(This is why there is a Tor Browser.).
This tiny change lets me put proper hidden service configuration into
single `i2p.go` file which lives in `modules/graceful/` and which never
has to be checked in to your codebase or affect your dependencies or
bloat your project in any way, it can live on a branch in my fork and
I'll fast-forward every release and never the twain shall meet.
The main use-case for this is to listen on Peer-to-Peer networks and
Hidden Services directly without error-prone and cumbersome
port-forwarding configuration. For instance, I might implement an
"I2PGetListener" as follows:
```Go
// adapted from i2p.go which is unchecked-in in my modules/graceful/ directory
import "github.com/eyedeekay/onramp"
var garlic = &onramp.Garlic{}
func I2PGetListener(network, address string) (net.Listener, error) {
// Add a deferral to say that we've tried to grab a listener
defer GetManager().InformCleanup()
switch network {
case "tcp", "tcp4", "tcp6", "i2p", "i2pt":
return garlic.Listen()
case "unix", "unixpacket":
// I2P isn't really a replacement for the stuff you use Unix sockets for and it's also not an anonymity risk, so treat them normally
unixAddr, err := net.ResolveUnixAddr(network, address)
if err != nil {
return nil, err
}
return GetListenerUnix(network, unixAddr)
default:
return nil, net.UnknownNetworkError(network)
}
}
```
I could then substitute that GetListener function and be 50% of the way
to having a fully-functioning gitea-over-hidden-services instance
without any additional configuration(The other 50% doesn't require any
code-changes on gitea's part).
There are 2 advantages here, one being convenience, first this turns
hidden services into a zero-configuration option for self-hosting gitea,
and second safety, these Go libraries are passing around
hidden-service-only versions of the net.Addr struct, they're using
hidden-service-only versions of the sockets, which are both expressly
designed to never require access to any information outside the hidden
service network, manipulating the application so it reveals information
about the host becomes much more difficult, and some attacks become
nearly impossible. It also opens up TLS-over-Hidden Services support
which is niche right now, of course, but in a future where gitea
instances federate if hidden services want to be part of the federation
they're probably going to need TLS certificates. They don't need to be
painful to set up.
This doesn't fix an open issue, but it might affect:
- https://github.com/go-gitea/gitea/issues/22335 - my `i2p.go` file
actually has a mod that fixes this but it requires adding a handful of
new dependencies to gitea and isn't compatible with the normal way you
guys recommend using a proxy so I don't think it's ready to send to you
as a PR, but if I can find a non-invasive way to fix it I will.
- https://github.com/go-gitea/gitea/issues/18240
I hereby agree to the Code of Conduct published here:
8b89563bf1/CODE_OF_CONDUCT.md
I have read and understood the recommendations published here:
8b89563bf1/CONTRIBUTING.md
Thank you for your consideration.
---------
Co-authored-by: eyedeekay <idk@mulder>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
If the log file can't be opened, what should it do? panic/exit? ignore
logs? fallback to stderr?
It seems that "fallback to stderr" is slightly better than others ....
To avoid deadlock problem, almost database related functions should be
have ctx as the first parameter.
This PR do a refactor for some of these functions.
Replace #25892
Close #21942
Close #25464
Major changes:
1. Serve "robots.txt" and ".well-known/security.txt" in the "public"
custom path
* All files in "public/.well-known" can be served, just like
"public/assets"
3. Add a test for ".well-known/security.txt"
4. Simplify the "FileHandlerFunc" logic, now the paths are consistent so
the code can be simpler
5. Add CORS header for ".well-known" endpoints
6. Add logs to tell users they should move some of their legacy custom
public files
```
2023/07/19 13:00:37 cmd/web.go:178:serveInstalled() [E] Found legacy public asset "img" in CustomPath. Please move it to /work/gitea/custom/public/assets/img
2023/07/19 13:00:37 cmd/web.go:182:serveInstalled() [E] Found legacy public asset "robots.txt" in CustomPath. Please move it to /work/gitea/custom/public/robots.txt
```
This PR is not breaking.
---------
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Giteabot <teabot@gitea.io>
Fix#25776. Close#25826.
In the discussion of #25776, @wolfogre's suggestion was to remove the
commit status of `running` and `warning` to keep it consistent with
github.
references:
-
https://docs.github.com/en/rest/commits/statuses?apiVersion=2022-11-28#about-commit-statuses
## ⚠️ BREAKING ⚠️
So the commit status of Gitea will be consistent with GitHub, only
`pending`, `success`, `error` and `failure`, while `warning` and
`running` are not supported anymore.
---------
Co-authored-by: Jason Song <i@wolfogre.com>
The package `github.com/nfnt/resize` is deprecated and archived by the
author. `github.com/oliamb/cutter` is not maintained since 2018. We
could use `golang.org/x/image/draw` instead.
Bumping `github.com/golang-jwt/jwt` from v4 to v5.
`github.com/golang-jwt/jwt` v5 is bringing some breaking changes:
- standard `Valid()` method on claims is removed. It's replaced by
`ClaimsValidator` interface implementing `Validator()` method instead,
which is called after standard validation. Gitea doesn't seem to be
using this logic.
- `jwt.Token` has a field `Valid`, so it's checked in `ParseToken`
function in `services/auth/source/oauth2/token.go`
---------
Co-authored-by: Giteabot <teabot@gitea.io>
Update WorkPath/WORK_PATH related documents, remove out-dated
information.
Remove "StaticRootPath" on the admin config display page, because few
end user really need it, it only causes misconfiguration.
Co-authored-by: Giteabot <teabot@gitea.io>
Before: the concept "Content string" is used everywhere. It has some
problems:
1. Sometimes it means "base64 encoded content", sometimes it means "raw
binary content"
2. It doesn't work with large files, eg: uploading a 1G LFS file would
make Gitea process OOM
This PR does the refactoring: use "ContentReader" / "ContentBase64"
instead of "Content"
This PR is not breaking because the key in API JSON is still "content":
`` ContentBase64 string `json:"content"` ``
Move `public/*` to `public/assets/*`
Some old PRs (like #15219) introduced inconsistent directory system.
For example: why the local directory "public" is accessed by
`http://site/assets`? How to serve the ".well-known" files properly in
the public directory?
For convention rules, the "public" directory is widely used for the
website's root directory. It shouldn't be an exception for Gitea.
So, this PR makes the things consistent:
* `http://site/assets/foo` means `{CustomPath}/public/assets/foo`.
* `{CustomPath}/public/.well-known` and `{CustomPath}/public/robots.txt`
can be used in the future.
This PR is also a prerequisite for a clear solution for:
* #21942
* #25892
* discourse.gitea.io: [.well-known path serving custom files behind
proxy?](https://discourse.gitea.io/t/well-known-path-serving-custom-files-behind-proxy/5445/1)
This PR is breaking for users who have custom "public" files (CSS/JS).
After getting approvals, I will update the documents.
----
## ⚠️ BREAKING ⚠️
If you have files in your "custom/public/" folder, please move them to
"custom/public/assets/".
---------
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Giteabot <teabot@gitea.io>
Regression: https://github.com/go-gitea/gitea/pull/24805Closes: #25945
- Disallow `javascript`, `vbscript` and `data` (data uri images still
work) url schemes even if all other schemes are allowed
- Fixed older `cbthunderlink` tests
---------
Co-authored-by: delvh <dev.lh@web.de>
The "creating dir/file during load config" is a longstanding and complex
problem.
This PR only does a quick patch, it still needs more refactorings in the
future.
Fix#25938
Before, in dev mode, there might be some error logs like:
```
2023/07/17 13:54:51 ...s/assetfs/layered.go:221:WatchLocalChanges() [E] Unable to watch directory .: lstat /data/work/gitea/custom/templates: no such file or directory
```
Because there is no "custom/templates" directory.
After: ignore such error, no such error message anymore.
Got the same problem as #25915 when updating an instance. The
`log.Fatal` should have been marked as breaking in #23911.
This PR adds a notice that the system is shutting down because of the
deprecated setting.
The `FileBlame` function looks strange, it has `revision` as argument
but doesn't use it.
Since the function never be used, I think we could just remove it.
If anyone thinks it should be kept, please help fix `revision`.
Co-authored-by: Giteabot <teabot@gitea.io>
Replace #25580Fix#19453
The problem was: when users set "GITEA__XXX__YYY" , the "install page"
doesn't respect it.
So, to make the result consistent and avoid surprising end users, now
the "install page" also writes the environment variables to the config
file.
And, to make things clear, there are enough messages on the UI to tell
users what will happen.
There are some necessary/related changes to `environment-to-ini.go`:
* The "--clear" flag is removed and it was incorrectly written there.
The "clear" operation should be done if INSTALL_LOCK=true
* The "--prefix" flag is removed because it's never used, never
documented and it only causes inconsistent behavior.
Fixes (?) #25538
Fixes https://codeberg.org/forgejo/forgejo/issues/972
Regression #23879#23879 introduced a change which prevents read access to packages if a
user is not a member of an organization.
That PR also contained a change which disallows package access if the
team unit is configured with "no access" for packages. I don't think
this change makes sense (at the moment). It may be relevant for private
orgs. But for public or limited orgs that's useless because an
unauthorized user would have more access rights than the team member.
This PR restores the old behaviour "If a user has read access for an
owner, they can read packages".
---------
Co-authored-by: Giteabot <teabot@gitea.io>
To mock a handler:
```go
web.RouteMock(web.MockAfterMiddlewares, func(ctx *context.Context) {
// ...
})
defer web.RouteMockReset()
```
It helps:
* Test the middleware's behavior (assert the ctx.Data, etc)
* Mock the middleware's behavior (prepare some context data for handler)
* Mock the handler's response for some test cases, especially for some
integration tests and e2e tests.
Follow #25229
At present, when the trigger event is `pull_request_target`, the `ref`
and `sha` of `ActionRun` are set according to the base branch of the
pull request. This makes it impossible for us to find the head branch of
the `ActionRun` directly. In this PR, the `ref` and `sha` will always be
set to the head branch and they will be changed to the base branch when
generating the task context.
A couple of notes:
* Future changes should refactor arguments into a struct
* This filtering only is supported by meilisearch right now
* Issue index number is bumped which will cause a re-index
Fix#25736
Caused by #24048
Right now we only check the activity type for `pull_request` event when
`types` is specified or there are no `types` and filter. If a workflow
only specifies filters but no `types` like this:
```
on:
pull_request:
branches: [main]
```
the workflow will be triggered even if the activity type is not one of
`[opened, reopened, sync]`. We need to check the activity type in this
case.
Co-authored-by: Giteabot <teabot@gitea.io>
Remove unnecessary `if opts.Logger != nil` checks.
* For "CLI doctor" mode, output to the console's "logger.Info".
* For "Web Task" mode, output to the default "logger.Debug", to avoid
flooding the server's log in a busy production instance.
Co-authored-by: Giteabot <teabot@gitea.io>
To record which command is slow, this PR adds a debug log for slow git
operations.
---------
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: delvh <dev.lh@web.de>
Fix regression of #5363 (so long ago).
The old code definded a document mapping for `issueIndexerDocType`, and
assigned it to `BleveIndexerData` as its type. (`BleveIndexerData` has
been renamed to `IndexerData` in #25174, but nothing more.) But the old
code never used `BleveIndexerData`, it wrote the index with an anonymous
struct type. Nonetheless, bleve would use the default auto-mapping for
struct it didn't know, so the indexer still worked. This means the
custom document mapping was always dead code.
The custom document mapping is not useless, it can reduce index storage,
this PR brings it back and disable default mapping to prevent it from
happening again. Since `IndexerData`(`BleveIndexerData`) has JSON tags,
and bleve uses them first, so we should use `repo_id` as the field name
instead of `RepoID`.
I did a test to compare the storage size before and after this, with
about 3k real comments that were migrated from some public repos.
Before:
```text
[ 160] .
├── [ 42] index_meta.json
├── [ 13] rupture_meta.json
└── [ 128] store
├── [6.9M] 00000000005d.zap
└── [256K] root.bolt
```
After:
```text
[ 160] .
├── [ 42] index_meta.json
├── [ 13] rupture_meta.json
└── [ 128] store
├── [3.5M] 000000000065.zap
└── [256K] root.bolt
```
It saves about half the storage space.
---------
Co-authored-by: Giteabot <teabot@gitea.io>
Fixes#24723
Direct serving of content aka HTTP redirect is not mentioned in any of
the package registry specs but lots of official registries do that so it
should be supported by the usual clients.
This prevents the disk from overflowing with auth keys file
Fixes#17117
## ⚠️ BREAKING
This changes the default option for creating a backup of the authorized
key file when an update is made to default to false.
When branch's commit CommitMessage is too long, the column maybe too
short.(TEXT 16K for mysql).
This PR will fix it to only store the summary because these message will
only show on branch list or possible future search?
Resolve#24789
## ⚠️ BREAKING ⚠️
Before this, `DEFAULT_ACTIONS_URL` cound be set to any custom URLs like
`https://gitea.com` or `http://your-git-server,https://gitea.com`, and
the default value was `https://gitea.com`.
But now, `DEFAULT_ACTIONS_URL` supports only
`github`(`https://github.com`) or `self`(the root url of current Gitea
instance), and the default value is `github`.
If it has configured with a URL, an error log will be displayed and it
will fallback to `github`.
Actually, what we really want to do is always make it
`https://github.com`, however, this may not be acceptable for some
instances of internal use, so there's extra support for `self`, but no
more, even `https://gitea.com`.
Please note that `uses: https://xxx/yyy/zzz` always works and it does
exactly what it is supposed to do.
Although it's breaking, I belive it should be backported to `v1.20` due
to some security issues.
Follow-up on the runner side:
- https://gitea.com/gitea/act_runner/pulls/262
- https://gitea.com/gitea/act/pulls/70
This adds an API for uploading and Deleting Avatars for of Users, Repos
and Organisations. I'm not sure, if this should also be added to the
Admin API.
Resolves#25344
---------
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Giteabot <teabot@gitea.io>
Related #14180
Related #25233
Related #22639Close#19786
Related #12763
This PR will change all the branches retrieve method from reading git
data to read database to reduce git read operations.
- [x] Sync git branches information into database when push git data
- [x] Create a new table `Branch`, merge some columns of `DeletedBranch`
into `Branch` table and drop the table `DeletedBranch`.
- [x] Read `Branch` table when visit `code` -> `branch` page
- [x] Read `Branch` table when list branch names in `code` page dropdown
- [x] Read `Branch` table when list git ref compare page
- [x] Provide a button in admin page to manually sync all branches.
- [x] Sync branches if repository is not empty but database branches are
empty when visiting pages with branches list
- [x] Use `commit_time desc` as the default FindBranch order by to keep
consistent as before and deleted branches will be always at the end.
---------
Co-authored-by: Jason Song <i@wolfogre.com>
releated to #21820
- Split `Size` in repository table as two new colunms, one is `GitSize`
for git size, the other is `LFSSize` for lfs data. still store full size
in `Size` colunm.
- Show full size on ui, but show each of them by a `title`; example:
- Return full size in api response.
---------
Signed-off-by: a1012112796 <1012112796@qq.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: DmitryFrolovTri <23313323+DmitryFrolovTri@users.noreply.github.com>
Co-authored-by: Giteabot <teabot@gitea.io>
Fix#25451.
Bugfixes:
- When stopping the zombie or endless tasks, set `LogInStorage` to true
after transferring the file to storage. It was missing, it could write
to a nonexistent file in DBFS because `LogInStorage` was false.
- Always update `ActionTask.Updated` when there's a new state reported
by the runner, even if there's no change. This is to avoid the task
being judged as a zombie task.
Enhancement:
- Support `Stat()` for DBFS file.
- `WriteLogs` refuses to write if it could result in content holes.
---------
Co-authored-by: Giteabot <teabot@gitea.io>
More fix for #24981
* #24981Close#22361
* #22361
There were many patches for Gitea's sub-commands to satisfy the facts:
* Some sub-commands shouldn't output any log, otherwise the git protocol
would be broken
* Sometimes the users want to see "verbose" or "quiet" outputs
That's a longstanding problem, and very fragile. This PR is only a quick
patch for the problem.
In the future, the sub-command system should be refactored to a clear
solution.
----
Other changes:
* Use `ReplaceAllWriters` to replace
`RemoveAllWriters().AddWriters(writer)`, then it's an atomic operation.
* Remove unnecessary `syncLevelInternal` calls, because
`AddWriters/addWritersInternal` already calls it.
Co-authored-by: Giteabot <teabot@gitea.io>
Fix#25088
This PR adds the support for
[`pull_request_target`](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#pull_request_target)
workflow trigger. `pull_request_target` is similar to `pull_request`,
but the workflow triggered by the `pull_request_target` event runs in
the context of the base branch of the pull request rather than the head
branch. Since the workflow from the base is considered trusted, it can
access the secrets and doesn't need approvals to run.
Fix#25481
The `InitWorkPathAndCommonConfig` calls `LoadCommonSettings` which does
many checks like "current user is root or not".
Some commands like "environment-to-ini" shouldn't do such check, because
it might be run with "root" user at the moment (eg: the docker's setup
script)
ps: in the future, the docker's setup script should be improved to avoid
Gitea's command running with "root"
Fix regression of #25174.
The `Init` of the db indexer should return true to indicate that the
index was opened/existed, or the indexer will try to populate the index
(not really populate, just fill the queue, `Index` method of the db
indexer is a dummy).
Refactor `modules/indexer` to make it more maintainable. And it can be
easier to support more features. I'm trying to solve some of issue
searching, this is a precursor to making functional changes.
Current supported engines and the index versions:
| engines | issues | code |
| - | - | - |
| db | Just a wrapper for database queries, doesn't need version | - |
| bleve | The version of index is **2** | The version of index is **6**
|
| elasticsearch | The old index has no version, will be treated as
version **0** in this PR | The version of index is **1** |
| meilisearch | The old index has no version, will be treated as version
**0** in this PR | - |
## Changes
### Split
Splited it into mutiple packages
```text
indexer
├── internal
│ ├── bleve
│ ├── db
│ ├── elasticsearch
│ └── meilisearch
├── code
│ ├── bleve
│ ├── elasticsearch
│ └── internal
└── issues
├── bleve
├── db
├── elasticsearch
├── internal
└── meilisearch
```
- `indexer/interanal`: Internal shared package for indexer.
- `indexer/interanal/[engine]`: Internal shared package for each engine
(bleve/db/elasticsearch/meilisearch).
- `indexer/code`: Implementations for code indexer.
- `indexer/code/internal`: Internal shared package for code indexer.
- `indexer/code/[engine]`: Implementation via each engine for code
indexer.
- `indexer/issues`: Implementations for issues indexer.
### Deduplication
- Combine `Init/Ping/Close` for code indexer and issues indexer.
- ~Combine `issues.indexerHolder` and `code.wrappedIndexer` to
`internal.IndexHolder`.~ Remove it, use dummy indexer instead when the
indexer is not ready.
- Duplicate two copies of creating ES clients.
- Duplicate two copies of `indexerID()`.
### Enhancement
- [x] Support index version for elasticsearch issues indexer, the old
index without version will be treated as version 0.
- [x] Fix spell of `elastic_search/ElasticSearch`, it should be
`Elasticsearch`.
- [x] Improve versioning of ES index. We don't need `Aliases`:
- Gitea does't need aliases for "Zero Downtime" because it never delete
old indexes.
- The old code of issues indexer uses the orignal name to create issue
index, so it's tricky to convert it to an alias.
- [x] Support index version for meilisearch issues indexer, the old
index without version will be treated as version 0.
- [x] Do "ping" only when `Ping` has been called, don't ping
periodically and cache the status.
- [x] Support the context parameter whenever possible.
- [x] Fix outdated example config.
- [x] Give up the requeue logic of issues indexer: When indexing fails,
call Ping to check if it was caused by the engine being unavailable, and
only requeue the task if the engine is unavailable.
- It is fragile and tricky, could cause data losing (It did happen when
I was doing some tests for this PR). And it works for ES only.
- Just always requeue the failed task, if it caused by bad data, it's a
bug of Gitea which should be fixed.
---------
Co-authored-by: Giteabot <teabot@gitea.io>
this will allow us to fully localize it later
PS: we can not migrate back as the old value was a one-way conversion
prepare for #25213
---
*Sponsored by Kithara Software GmbH*
In modern days, there is no reason to make users set "charset" anymore.
Close#25378
## ⚠️ BREAKING
The key `[database].CHARSET` was removed completely as every newer
(>10years) MySQL database supports `utf8mb4` already.
There is a (deliberately) undocumented new fallback option if anyone
still needs to use it, but we don't recommend using it as it simply
causes problems.
# The problem
There were many "path tricks":
* By default, Gitea uses its program directory as its work path
* Gitea tries to use the "work path" to guess its "custom path" and
"custom conf (app.ini)"
* Users might want to use other directories as work path
* The non-default work path should be passed to Gitea by GITEA_WORK_DIR
or "--work-path"
* But some Gitea processes are started without these values
* The "serv" process started by OpenSSH server
* The CLI sub-commands started by site admin
* The paths are guessed by SetCustomPathAndConf again and again
* The default values of "work path / custom path / custom conf" can be
changed when compiling
# The solution
* Use `InitWorkPathAndCommonConfig` to handle these path tricks, and use
test code to cover its behaviors.
* When Gitea's web server runs, write the WORK_PATH to "app.ini", this
value must be the most correct one, because if this value is not right,
users would find that the web UI doesn't work and then they should be
able to fix it.
* Then all other sub-commands can use the WORK_PATH in app.ini to
initialize their paths.
* By the way, when Gitea starts for git protocol, it shouldn't output
any log, otherwise the git protocol gets broken and client blocks
forever.
The "work path" priority is: WORK_PATH in app.ini > cmd arg --work-path
> env var GITEA_WORK_DIR > builtin default
The "app.ini" searching order is: cmd arg --config > cmd arg "work path
/ custom path" > env var "work path / custom path" > builtin default
## ⚠️ BREAKING
If your instance's "work path / custom path / custom conf" doesn't meet
the requirements (eg: work path must be absolute), Gitea will report a
fatal error and exit. You need to set these values according to the
error log.
----
Close#24818Close#24222Close#21606Close#21498Close#25107Close#24981
Maybe close#24503
Replace #23301
Replace #22754
And maybe more
That's a longstanding INI package problem: the "MustXxx" calls change
the option values, and the following "Save" will save a lot of garbage
options into the user's config file.
Ideally we should refactor the INI package to a clear solution, but it's
a huge work.
A clear workaround is what this PR does: when "Save", load a clear INI
instance and save it.
Partially fix#25377, the "install" page needs more fine tunes.
### Summary
Extend the template variable substitution to replace file paths. This
can be helpful for setting up log files & directories that should match
the repository name.
### PR Changes
- Move files matching glob pattern when setting up repos from template
- For security, added ~escaping~ sanitization for cross-platform support
and to prevent directory traversal (thanks @silverwind for the
reference)
- Added unit testing for escaping function
- Fixed the integration tests for repo template generation by passing
the repo_template_id
- Updated the integration testfiles to add some variable substitution &
assert the outputs
I had to fix the existing repo template integration test and extend it
to add a check for variable substitutions.
Example:
1. The "web" package shouldn't depends on "modules/context" package,
instead, let each "web context" register themselves to the "web"
package.
2. The old Init/Free doesn't make sense, so simplify it
* The ctx in "Init(ctx)" is never used, and shouldn't be used that way
* The "Free" is never called and shouldn't be called because the SSPI
instance is shared
---------
Co-authored-by: Giteabot <teabot@gitea.io>
Follow up #22405Fix#20703
This PR rewrites storage configuration read sequences with some breaks
and tests. It becomes more strict than before and also fixed some
inherit problems.
- Move storage's MinioConfig struct into setting, so after the
configuration loading, the values will be stored into the struct but not
still on some section.
- All storages configurations should be stored on one section,
configuration items cannot be overrided by multiple sections. The
prioioty of configuration is `[attachment]` > `[storage.attachments]` |
`[storage.customized]` > `[storage]` > `default`
- For extra override configuration items, currently are `SERVE_DIRECT`,
`MINIO_BASE_PATH`, `MINIO_BUCKET`, which could be configured in another
section. The prioioty of the override configuration is `[attachment]` >
`[storage.attachments]` > `default`.
- Add more tests for storages configurations.
- Update the storage documentations.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
The plan is that all built-in auth providers use inline SVG for more
flexibility in styling and to get the GitHub icon to follow
`currentcolor`. This only removes the `public/img/auth` directory and
adds the missing svgs to our svg build.
It should map the built-in providers to these SVGs and render them. If
the user has set a Icon URL, it should render that as an `img` tag
instead.
```
gitea-azure-ad
gitea-bitbucket
gitea-discord
gitea-dropbox
gitea-facebook
gitea-gitea
gitea-gitlab
gitea-google
gitea-mastodon
gitea-microsoftonline
gitea-nextcloud
gitea-twitter
gitea-yandex
octicon-mark-github
```
GitHub logo is now white again on dark theme:
<img width="431" alt="Screenshot 2023-06-12 at 21 45 34"
src="https://github.com/go-gitea/gitea/assets/115237/27a43504-d60a-4132-a502-336b25883e4d">
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Fixes#25160.
`data-source-position` of checkboxes in a task list was incorrect
whenever there was YAML front matter. This would result in issue content
or PR descriptions getting corrupted with random `x` or space characters
when a user checked or unchecked a task.
If redirect_to parameter has set value starting with `\\example.com`
redirect will be created with header `Location: /\\example.com` that
will redirect to example.com domain.
Fixes some issues with the swagger documentation for the new multiple
files API endpoint (#24887) which were overlooked when submitting the
original PR:
1. add some missing parameter descriptions
2. set correct `required` option for required parameters
3. change endpoint description to match it full functionality (every
kind of file modification is supported, not just creating and updating)
## Changes
- Adds the following high level access scopes, each with `read` and
`write` levels:
- `activitypub`
- `admin` (hidden if user is not a site admin)
- `misc`
- `notification`
- `organization`
- `package`
- `issue`
- `repository`
- `user`
- Adds new middleware function `tokenRequiresScopes()` in addition to
`reqToken()`
- `tokenRequiresScopes()` is used for each high-level api section
- _if_ a scoped token is present, checks that the required scope is
included based on the section and HTTP method
- `reqToken()` is used for individual routes
- checks that required authentication is present (but does not check
scope levels as this will already have been handled by
`tokenRequiresScopes()`
- Adds migration to convert old scoped access tokens to the new set of
scopes
- Updates the user interface for scope selection
### User interface example
<img width="903" alt="Screen Shot 2023-05-31 at 1 56 55 PM"
src="https://github.com/go-gitea/gitea/assets/23248839/654766ec-2143-4f59-9037-3b51600e32f3">
<img width="917" alt="Screen Shot 2023-05-31 at 1 56 43 PM"
src="https://github.com/go-gitea/gitea/assets/23248839/1ad64081-012c-4a73-b393-66b30352654c">
## tokenRequiresScopes Design Decision
- `tokenRequiresScopes()` was added to more reliably cover api routes.
For an incoming request, this function uses the given scope category
(say `AccessTokenScopeCategoryOrganization`) and the HTTP method (say
`DELETE`) and verifies that any scoped tokens in use include
`delete:organization`.
- `reqToken()` is used to enforce auth for individual routes that
require it. If a scoped token is not present for a request,
`tokenRequiresScopes()` will not return an error
## TODO
- [x] Alphabetize scope categories
- [x] Change 'public repos only' to a radio button (private vs public).
Also expand this to organizations
- [X] Disable token creation if no scopes selected. Alternatively, show
warning
- [x] `reqToken()` is missing from many `POST/DELETE` routes in the api.
`tokenRequiresScopes()` only checks that a given token has the correct
scope, `reqToken()` must be used to check that a token (or some other
auth) is present.
- _This should be addressed in this PR_
- [x] The migration should be reviewed very carefully in order to
minimize access changes to existing user tokens.
- _This should be addressed in this PR_
- [x] Link to api to swagger documentation, clarify what
read/write/delete levels correspond to
- [x] Review cases where more than one scope is needed as this directly
deviates from the api definition.
- _This should be addressed in this PR_
- For example:
```go
m.Group("/users/{username}/orgs", func() {
m.Get("", reqToken(), org.ListUserOrgs)
m.Get("/{org}/permissions", reqToken(), org.GetUserOrgsPermissions)
}, tokenRequiresScopes(auth_model.AccessTokenScopeCategoryUser,
auth_model.AccessTokenScopeCategoryOrganization),
context_service.UserAssignmentAPI())
```
## Future improvements
- [ ] Add required scopes to swagger documentation
- [ ] Redesign `reqToken()` to be opt-out rather than opt-in
- [ ] Subdivide scopes like `repository`
- [ ] Once a token is created, if it has no scopes, we should display
text instead of an empty bullet point
- [ ] If the 'public repos only' option is selected, should read
categories be selected by default
Closes#24501Closes#24799
Co-authored-by: Jonathan Tran <jon@allspice.io>
Co-authored-by: Kyle D <kdumontnu@gmail.com>
Co-authored-by: silverwind <me@silverwind.io>
Before, Gitea shows the database table stats on the `admin dashboard`
page.
It has some problems:
* `count(*)` is quite heavy. If tables have many records, this blocks
loading the admin page blocks for a long time
* Some users had even reported issues that they can't visit their admin
page because this page causes blocking or `50x error (reverse proxy
timeout)`
* The `actions` stat is not useful. The table is simply too large. Does
it really matter if it contains 1,000,000 rows or 9,999,999 rows?
* The translation `admin.dashboard.statistic_info` is difficult to
maintain.
So, this PR uses a separate page to show the stats and removes the
`actions` stat.
## ⚠️ BREAKING
The `actions` Prometheus metrics collector has been removed for the
reasons mentioned beforehand.
Please do not rely on its output anymore.
The INI package has many bugs and quirks, and in fact it is
unmaintained.
This PR is the first step for the INI package refactoring:
* Use Gitea's "config_provider" to provide INI access
* Deprecate the INI package by golangci.yml rule
It's been disabled by default since 1.17
(https://github.com/go-gitea/gitea/pull/18914), and it never really
delivered any benefit except being another cache layer that has its own
unsolved invalidation issues. HTTP cache works, we don't need two cache
layers at the browser for assets.
## ⚠️ BREAKING
You can remove the config `[ui].USE_SERVICE_WORKER` from your `app.ini`
now.
The admin config page has been broken for many many times, a little
refactoring would make this page panic.
So, add a test for it, and add another test to cover the 500 error page.
Co-authored-by: Giteabot <teabot@gitea.io>
This PR creates an API endpoint for creating/updating/deleting multiple
files in one API call similar to the solution provided by
[GitLab](https://docs.gitlab.com/ee/api/commits.html#create-a-commit-with-multiple-files-and-actions).
To archive this, the CreateOrUpdateRepoFile and DeleteRepoFIle functions
in files service are unified into one function supporting multiple files
and actions.
Resolves#14619
gitea.com experienced the corrupted LevelQueue bug again.
I think the problem is clear now: if the keys in LevelDB went
out-of-sync, the LevelQueue itself doesn't have the ability to recover,
eg:
* LevelQueue.Len() reports 100
* LevelQueue.LPop() reports ErrNotFound = errors.New("no key found")
So it needs to dive into the LevelDB to remove all keys to recover the
corrupted LevelQueue.
More comments are in TestCorruptedLevelQueue.
As discussed in
https://github.com/go-gitea/gitea/pull/24953#issuecomment-1565630156.
## ⚠️ BREAKING ⚠️
1. The `ui.THEME_COLOR_META_TAG` setting has been removed. If you still
need to set the `theme-color` meta tag, add it via
`$GITEA_CUSTOM/templates/custom/header.tmpl` instead.
2. The non-standard `default-theme` meta-tag added in
https://github.com/go-gitea/gitea/pull/13809 has been removed. Third
party code that needs to obtain the currently loaded theme should use
the `theme-<name>` class on the `<html>` node instead, which reflect the
currently active theme.
The old EventWriter's Run does:
```go
for {
handlePause()
select {
case event <- Queue:
write the log event ...
}
}
```
So, if an event writer is started before the logger is paused, there is
a chance that the logger isn't paused for the first message.
The new logic is:
```go
for {
select {
case event <- Queue:
handlePause()
write the log event ...
}
}
```
Then the event writer can be correctly paused
The old process manager's `Trace` function by default calls `log.Printf`
to output "trace" level logs. That's not ideal because by default the
trace level logs should not be outputted. In history it didn't cause
problems because there was no other call to the process manager before
the logger system's initialization.
But if there is any package using the process manager before the "Trace"
function gets assigned to the logger system's trace function, the
process manager will outputs unexpected verbose messages, this behavior
is not expected in most cases.
Now, the logger system also uses process manager to manage its goroutine
contexts, so it's the time to fix the old "trace" behavior: by default,
do not output the trace level messages. Fix#24951
Before there was a "graceful function": RunWithShutdownFns, it's mainly
for some modules which doesn't support context.
The old queue system doesn't work well with context, so the old queues
need it.
After the queue refactoring, the new queue works with context well, so,
use Golang context as much as possible, the `RunWithShutdownFns` could
be removed (replaced by RunWithCancel for context cancel mechanism), the
related code could be simplified.
This PR also fixes some legacy queue-init problems, eg:
* typo : archiver: "unable to create codes indexer queue" => "unable to
create repo-archive queue"
* no nil check for failed queues, which causes unfriendly panic
After this PR, many goroutines could have better display name:
This PR replaces all string refName as a type `git.RefName` to make the
code more maintainable.
Fix#15367
Replaces #23070
It also fixed a bug that tags are not sync because `git remote --prune
origin` will not remove local tags if remote removed.
We in fact should use `git fetch --prune --tags origin` but not `git
remote update origin` to do the sync.
Some answer from ChatGPT as ref.
> If the git fetch --prune --tags command is not working as expected,
there could be a few reasons why. Here are a few things to check:
>
>Make sure that you have the latest version of Git installed on your
system. You can check the version by running git --version in your
terminal. If you have an outdated version, try updating Git and see if
that resolves the issue.
>
>Check that your Git repository is properly configured to track the
remote repository's tags. You can check this by running git config
--get-all remote.origin.fetch and verifying that it includes
+refs/tags/*:refs/tags/*. If it does not, you can add it by running git
config --add remote.origin.fetch "+refs/tags/*:refs/tags/*".
>
>Verify that the tags you are trying to prune actually exist on the
remote repository. You can do this by running git ls-remote --tags
origin to list all the tags on the remote repository.
>
>Check if any local tags have been created that match the names of tags
on the remote repository. If so, these local tags may be preventing the
git fetch --prune --tags command from working properly. You can delete
local tags using the git tag -d command.
---------
Co-authored-by: delvh <dev.lh@web.de>
This adds the ability to pin important Issues and Pull Requests. You can
also move pinned Issues around to change their Position. Resolves#2175.
## Screenshots
The Design was mostly copied from the Projects Board.
## Implementation
This uses a new `pin_order` Column in the `issue` table. If the value is
set to 0, the Issue is not pinned. If it's set to a bigger value, the
value is the Position. 1 means it's the first pinned Issue, 2 means it's
the second one etc. This is dived into Issues and Pull requests for each
Repo.
## TODO
- [x] You can currently pin as many Issues as you want. Maybe we should
add a Limit, which is configurable. GitHub uses 3, but I prefer 6, as
this is better for bigger Projects, but I'm open for suggestions.
- [x] Pin and Unpin events need to be added to the Issue history.
- [x] Tests
- [x] Migration
**The feature itself is currently fully working, so tester who may find
weird edge cases are very welcome!**
---------
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Giteabot <teabot@gitea.io>
1. non-dev mode is treated as prod mode, to protect users from
accidentally running in dev mode if there is a typo in this value.
2. in dev mode, do not need to really exit if there are template errors,
because the template errors could be fixed by developer soon and the
templates get reloaded, help:
* https://github.com/go-gitea/gitea/issues/24845#issuecomment-1557615382
3. Fine tune the mail template loading message.
close https://github.com/go-gitea/gitea/issues/16321
Provided a webhook trigger for requesting someone to review the Pull
Request.
Some modifications have been made to the returned `PullRequestPayload`
based on the GitHub webhook settings, including:
- add a description of the current reviewer object as
`RequestedReviewer` .
- setting the action to either **review_requested** or
**review_request_removed** based on the operation.
- adding the `RequestedReviewers` field to the issues_model.PullRequest.
This field will be loaded into the PullRequest through
`LoadRequestedReviewers()` when `ToAPIPullRequest` is called.
After the Pull Request is merged, I will supplement the relevant
documentation.
Fix#24896
If users set different languages by `linguist-language`, the `stats` map
could be: `java: 100, Java: 200`.
Language stats are stored as case-insensitive in database and there is a
unique key.
So, the different language names should be merged to one unique name:
`Java: 300`
Replace #19857Close#19856Close#10311Close#10123
Major changes:
1. Move a lot of code from `environment-to-ini.go` to `config_env.go` to
make them testable.
2. Add `__FILE` support
3. Update documents
4. Add tests
Close#13454 , Close#23255, Close#14697 (and maybe more related
issues)
Many users have the requirement to customize the git config. This PR
introduces an easy way: put the options in Gitea's app.ini
`[git.config]`, then the config options will be applied to git config.
And it can support more flexible default config values, eg: now
`diff.algorithm=histogram` by default. According to:
https://stackoverflow.com/a/32367597/4754037 , `histogram diff` is
efficient and doesn't like to cause server-side problems.
---------
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: Giteabot <teabot@gitea.io>
Fix#24882
The goroutines are all asynchronized. So it needs a little "sleep" to
make sure the writer's goroutine has been paused before sending messages
to it.
Co-authored-by: Giteabot <teabot@gitea.io>
Fix#24856
Rename "context.contextKey" to "context.WebContextKey", this context is
for web context only. But the Context itself is not renamed, otherwise
it would cause a lot of changes (if we really want to rename it, there
could be a separate PR).
The old test code doesn't really test, the "install page" gets broken
not only one time, so use new test code to make sure the "install page"
could work.
Replace #20257 (which is stale and incomplete)
Close#20255
Major changes:
* Deprecate the "WHITELIST", use "ALLOWLIST"
* Add wildcard support for EMAIL_DOMAIN_ALLOWLIST/EMAIL_DOMAIN_BLOCKLIST
* Update example config file and document
* Improve tests
## ⚠️ Breaking
The `log.<mode>.<logger>` style config has been dropped. If you used it,
please check the new config manual & app.example.ini to make your
instance output logs as expected.
Although many legacy options still work, it's encouraged to upgrade to
the new options.
The SMTP logger is deleted because SMTP is not suitable to collect logs.
If you have manually configured Gitea log options, please confirm the
logger system works as expected after upgrading.
## Description
Close#12082 and maybe more log-related issues, resolve some related
FIXMEs in old code (which seems unfixable before)
Just like rewriting queue #24505 : make code maintainable, clear legacy
bugs, and add the ability to support more writers (eg: JSON, structured
log)
There is a new document (with examples): `logging-config.en-us.md`
This PR is safer than the queue rewriting, because it's just for
logging, it won't break other logic.
## The old problems
The logging system is quite old and difficult to maintain:
* Unclear concepts: Logger, NamedLogger, MultiChannelledLogger,
SubLogger, EventLogger, WriterLogger etc
* Some code is diffuclt to konw whether it is right:
`log.DelNamedLogger("console")` vs `log.DelNamedLogger(log.DEFAULT)` vs
`log.DelLogger("console")`
* The old system heavily depends on ini config system, it's difficult to
create new logger for different purpose, and it's very fragile.
* The "color" trick is difficult to use and read, many colors are
unnecessary, and in the future structured log could help
* It's difficult to add other log formats, eg: JSON format
* The log outputer doesn't have full control of its goroutine, it's
difficult to make outputer have advanced behaviors
* The logs could be lost in some cases: eg: no Fatal error when using
CLI.
* Config options are passed by JSON, which is quite fragile.
* INI package makes the KEY in `[log]` section visible in `[log.sub1]`
and `[log.sub1.subA]`, this behavior is quite fragile and would cause
more unclear problems, and there is no strong requirement to support
`log.<mode>.<logger>` syntax.
## The new design
See `logger.go` for documents.
## Screenshot
<details>
</details>
## TODO
* [x] add some new tests
* [x] fix some tests
* [x] test some sub-commands (manually ....)
---------
Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Giteabot <teabot@gitea.io>
This change makes the CSS for `<video>` in markup match that of `<img>`,
and also allows additional attributes to be used. This way the width,
padding, alignment should work equally well for both.
Replace #16455Close#21803
Mixing different Gitea contexts together causes some problems:
1. Unable to respond proper content when error occurs, eg: Web should
respond HTML while API should respond JSON
2. Unclear dependency, eg: it's unclear when Context is used in
APIContext, which fields should be initialized, which methods are
necessary.
To make things clear, this PR introduces a Base context, it only
provides basic Req/Resp/Data features.
This PR mainly moves code. There are still many legacy problems and
TODOs in code, leave unrelated changes to future PRs.
This PR
- [x] Move some functions from `issues.go` to `issue_stats.go` and
`issue_label.go`
- [x] Remove duplicated issue options `UserIssueStatsOption` to keep
only one `IssuesOptions`
Regression of #24536. If the user doesn't explicitly disable Actions, it
will be enabled.
1. Gitea will call `loadRepositoryFrom` before `loadActionsFrom`.
25d4f95df2/modules/setting/setting.go (L234-L237)
2. In `loadRepositoryFrom`,
`rootCfg.Section("actions").Key("ENABLED").MustBool(true)` will set
`actions.ENABLED` with `true`.
25d4f95df2/modules/setting/repository.go (L313-L315)
3. In `loadActionsFrom`, `rootCfg.Section("actions")` will get a section
with Actions enabled.
25d4f95df2/modules/setting/actions.go (L23-L26)
Although the cause of the problem was using `true` by copy-paste
mistake, it also surprised me that
**`rootCfg.Section("actions").Key("ENABLED").MustBool(true)` doesn't
only read, but also write.**
Make mailer SMTP check have timed context
Otherwise Gitea may block for long time if the DNS request blocks.
---------
Co-authored-by: Giteabot <teabot@gitea.io>
This PR adds support for the systemd notify protocol. Several status
messagess are provided. We should likely add a common notify/status
message for graceful.
Replaces #21140
Signed-off-by: Andrew Thornton <art27@cantab.net>
---------
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: ltdk <usr@ltdk.xyz>
Co-authored-by: Giteabot <teabot@gitea.io>
minio storage iterator shows different behavior with local fs iterator.
in local fs storage:
``` go
s.IterateObjects("prefix", func(path,obj)
println(path) // show "prefix/xxx.file"
})
```
in minio storage:
```go
s.IterateObjects("prefix", func(path,obj)
println(path) // show "xxx.file"
})
```
I think local fs is correct, minio use wrong `basePath` to trim storage
path prefix.
---------
Co-authored-by: Giteabot <teabot@gitea.io>
`http.ServeFile/ServeContent` handles `If-xxx`, `Content-Length`,
`Range` and `Etag` correctly
After this PR, storage files (eg: avatar) could be responded with
correct Content-Length.
fix#12192 Support SSH for go get
---------
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
Co-authored-by: mfk <mfk@hengwei.com.cn>
Co-authored-by: silverwind <me@silverwind.io>
Although some features are mixed together in this PR, this PR is not
that large, and these features are all related.
Actually there are more than 70 lines are for a toy "test queue", so
this PR is quite simple.
Major features:
1. Allow site admin to clear a queue (remove all items in a queue)
* Because there is no transaction, the "unique queue" could be corrupted
in rare cases, that's unfixable.
* eg: the item is in the "set" but not in the "list", so the item would
never be able to be pushed into the queue.
* Now site admin could simply clear the queue, then everything becomes
correct, the lost items could be re-pushed into queue by future
operations.
3. Split the "admin/monitor" to separate pages
4. Allow to download diagnosis report
* In history, there were many users reporting that Gitea queue gets
stuck, or Gitea's CPU is 100%
* With diagnosis report, maintainers could know what happens clearly
The diagnosis report sample:
[gitea-diagnosis-20230510-192913.zip](https://github.com/go-gitea/gitea/files/11441346/gitea-diagnosis-20230510-192913.zip)
, use "go tool pprof profile.dat" to view the report.
Screenshots:
---------
Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: Giteabot <teabot@gitea.io>
Replaces #24641
Currently, unit tests fail when run locally (unless users have minio
instance running). This PR only requires redis unit tests if in CI.
- Only run redis unit tests when `CI` env variable is set
- Add minio as a service in unit tests actions
According to the discussion with DanielGibson, the default "20m" seems
too short. It would make LFS fail if the file is large / network is
slow.
I think relaxing this timeout doesn't have side affect. So change the
default value to 24h, IMO that should be long enough.
## ⚠️ BREAKING
If admins want the previous timeout, they should set the setting
`[server].LFS_HTTP_AUTH_EXPIRY`.
#### Added
- API: Create a branch directly from commit on the create branch API
- Added `old_ref_name` parameter to allow creating a new branch from a
specific commit, tag, or branch.
- Deprecated `old_branch_name` parameter in favor of the new
`old_ref_name` parameter.
---------
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
1. Remove unused fields/methods in web context.
2. Make callers call target function directly instead of the light
wrapper like "IsUserRepoReaderSpecific"
3. The "issue template" code shouldn't be put in the "modules/context"
package, so move them to the service package.
---------
Co-authored-by: Giteabot <teabot@gitea.io>
# ⚠️ Breaking
Many deprecated queue config options are removed (actually, they should
have been removed in 1.18/1.19).
If you see the fatal message when starting Gitea: "Please update your
app.ini to remove deprecated config options", please follow the error
messages to remove these options from your app.ini.
Example:
```
2023/05/06 19:39:22 [E] Removed queue option: `[indexer].ISSUE_INDEXER_QUEUE_TYPE`. Use new options in `[queue.issue_indexer]`
2023/05/06 19:39:22 [E] Removed queue option: `[indexer].UPDATE_BUFFER_LEN`. Use new options in `[queue.issue_indexer]`
2023/05/06 19:39:22 [F] Please update your app.ini to remove deprecated config options
```
Many options in `[queue]` are are dropped, including:
`WRAP_IF_NECESSARY`, `MAX_ATTEMPTS`, `TIMEOUT`, `WORKERS`,
`BLOCK_TIMEOUT`, `BOOST_TIMEOUT`, `BOOST_WORKERS`, they can be removed
from app.ini.
# The problem
The old queue package has some legacy problems:
* complexity: I doubt few people could tell how it works.
* maintainability: Too many channels and mutex/cond are mixed together,
too many different structs/interfaces depends each other.
* stability: due to the complexity & maintainability, sometimes there
are strange bugs and difficult to debug, and some code doesn't have test
(indeed some code is difficult to test because a lot of things are mixed
together).
* general applicability: although it is called "queue", its behavior is
not a well-known queue.
* scalability: it doesn't seem easy to make it work with a cluster
without breaking its behaviors.
It came from some very old code to "avoid breaking", however, its
technical debt is too heavy now. It's a good time to introduce a better
"queue" package.
# The new queue package
It keeps using old config and concept as much as possible.
* It only contains two major kinds of concepts:
* The "base queue": channel, levelqueue, redis
* They have the same abstraction, the same interface, and they are
tested by the same testing code.
* The "WokerPoolQueue", it uses the "base queue" to provide "worker
pool" function, calls the "handler" to process the data in the base
queue.
* The new code doesn't do "PushBack"
* Think about a queue with many workers, the "PushBack" can't guarantee
the order for re-queued unhandled items, so in new code it just does
"normal push"
* The new code doesn't do "pause/resume"
* The "pause/resume" was designed to handle some handler's failure: eg:
document indexer (elasticsearch) is down
* If a queue is paused for long time, either the producers blocks or the
new items are dropped.
* The new code doesn't do such "pause/resume" trick, it's not a common
queue's behavior and it doesn't help much.
* If there are unhandled items, the "push" function just blocks for a
few seconds and then re-queue them and retry.
* The new code doesn't do "worker booster"
* Gitea's queue's handlers are light functions, the cost is only the
go-routine, so it doesn't make sense to "boost" them.
* The new code only use "max worker number" to limit the concurrent
workers.
* The new "Push" never blocks forever
* Instead of creating more and more blocking goroutines, return an error
is more friendly to the server and to the end user.
There are more details in code comments: eg: the "Flush" problem, the
strange "code.index" hanging problem, the "immediate" queue problem.
Almost ready for review.
TODO:
* [x] add some necessary comments during review
* [x] add some more tests if necessary
* [x] update documents and config options
* [x] test max worker / active worker
* [x] re-run the CI tasks to see whether any test is flaky
* [x] improve the `handleOldLengthConfiguration` to provide more
friendly messages
* [x] fine tune default config values (eg: length?)
## Code coverage:
The "modules/context.go" is too large to maintain.
This PR splits it to separate files, eg: context_request.go,
context_response.go, context_serve.go
This PR will help:
1. The future refactoring for Gitea's web context (eg: simplify the context)
2. Introduce proper "range request" support
3. Introduce context function
This PR only moves code, doesn't change any logic.
Due to #24409 , we can now specify '--not' when getting all commits from
a repo to exclude commits from a different branch.
When I wrote that PR, I forgot to also update the code that counts the
number of commits in the repo. So now, if the --not option is used, it
may return too many commits, which can indicate that another page of
data is available when it is not.
This PR passes --not to the commands that count the number of commits in
a repo
Help some users like #16832#1851
There are many users reporting similar problem: if the SECRET_KEY
mismatches, some operations (like 2FA login) only reports unclear 500
error and unclear "base64 decode error" log （some maintainers ever spent
a lot of time on debugging such problem)
The SECRET_KEY was not well-designed and it is also a kind of technical
debt. Since it couldn't be fixed easily, it's good to add clearer error
messages, then at least users could know what the real problem is.
---------
Co-authored-by: delvh <dev.lh@web.de>
This gives more "freshness" to the explore page. So it's not just the
same X users on the explore page by default, now it matches the same
sort as the repos on the explore page.
---------
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Replace #22117. Implement it in a more maintainable way.
Some licenses have placeholders e.g. the BSD licenses start with this
line:
```
Copyright (c) <year> <owner>.
```
This PR replaces the placeholders with the correct value when initialize
a new repo.
### FAQ
- Why not use a regex?
It will be a pretty complicated regex which could be hard to maintain.
- There're still missing placeholders.
There are over 500 licenses, it's impossible for anyone to inspect all
of them alone. Please help to add them if you find any, and it is also
OK to leave them for the future.
---------
Co-authored-by: Giteabot <teabot@gitea.io>
close#24449
The unit of `Actions` should be contorlled not only by
`repository.DISABLED_REPO_UNITS` but also by `actions.ENABLED`
in the `app.ini`.
Previously, the permission of the team's `Actions` unit was not
controlled by `actions.Enabled`. So, even if the user sets
`actions.Enabled` to false, he can still select the permission of the
`Actions` unit for the team.
This PR makes the permissions of the team's `Actions` unit also
controlled by `actions.Enabled`. Just append`TypeActions` into
`DisabledRepoUnits` slice when initializing if `actions.Enabled` is
false.
### Changes:
If `Actions` is set disbaled in `app.ini`, like below:
```yaml
[actions]
ENABLED = false
```
1. If user try to create/edit a team, will prompt user that `Actions` is disabled.
2. `actions` is not displayed in the sidebar on the team details page
Partially for #24457
Major changes:
1. The old `signedUserNameStringPointerKey` is quite hacky, use
`ctx.Data[SignedUser]` instead
2. Move duplicate code from `Contexter` to `CommonTemplateContextData`
3. Remove incorrect copying&pasting code `ctx.Data["Err_Password"] =
true` in API handlers
4. Use one unique `RenderPanicErrorPage` for panic error page rendering
5. Move `stripSlashesMiddleware` to be the first middleware
6. Install global panic recovery handler, it works for both `install`
and `web`
7. Make `500.tmpl` only depend minimal template functions/variables,
avoid triggering new panics
Screenshot:
<details>
</details>
The ordering of the final token causing a close of the queue in this
test may be out of sync due to concurrency. Instead just use ensure that
the queue is closed when everything expected is done.
Fixes: https://github.com/go-gitea/gitea/issues/23608
Fixes: https://github.com/go-gitea/gitea/issues/23977
---------
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
The old `HTMLRender` is not ideal.
1. It shouldn't be initialized multiple times, it consumes a lot of
memory and is slow.
2. It shouldn't depend on short-lived requests, the `WatchLocalChanges`
needs a long-running context.
3. It doesn't make sense to use FuncsMap slice.
HTMLRender was designed to only work for GItea's specialized 400+
templates, so it's good to make it a global shared instance.
For my specific use case, I'd like to get all commits that are on one
branch but NOT on the other branch.
For instance, I'd like to get all the commits on `Branch1` that are not
also on `master` (I.e. all commits that were made after `Branch1` was
created).
This PR adds a `not` query param that gets passed down to the `git log`
command to allow the user to exclude items from `GetAllCommits`.
See [git
documentation](https://git-scm.com/docs/git-log#Documentation/git-log.txt---not)
---------
Co-authored-by: Giteabot <teabot@gitea.io>
Co-authored-by: @awkwardbunny
This PR adds a Debian package registry. You can follow [this
tutorial](https://www.baeldung.com/linux/create-debian-package) to build
a *.deb package for testing. Source packages are not supported at the
moment and I did not find documentation of the architecture "all" and
how these packages should be treated.
---------
Co-authored-by: Brian Hong <brian@hongs.me>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
There was only one `IsRepositoryExist` function, it did: `has && isDir`
However it's not right, and it would cause 500 error when creating a new
repository if the dir exists.
Then, it was changed to `has || isDir`, it is still incorrect, it
affects the "adopt repo" logic.
To make the logic clear:
* IsRepositoryModelOrDirExist
* IsRepositoryModelExist
> The scoped token PR just checked all API routes but in fact, some web
routes like `LFS`, git `HTTP`, container, and attachments supports basic
auth. This PR added scoped token check for them.
---------
Signed-off-by: jolheiser <john.olheiser@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
## Needs and benefits
[Livebook](https://livebook.dev/) notebooks are used for code
documentation and for deep dives and note-taking in the elixir
ecosystem. Rendering these in these as Markdown on frogejo has many
benefits, since livemd is a subset of markdown. Some of the benefits
are:
- New users of elixir and livebook are scared by unformated .livemd
files, but are shown what they expect
- Sharing a notebook is as easy as sharing a link, no need to install
the software in order to see the results.
[goldmark-meraid ](https://github.com/abhinav/goldmark-mermaid) is a
mermaid-js parser already included in gitea. This makes the .livemd
rendering integration feature complete. With this PR class diagrams, ER
Diagrams, flow charts and much more will be rendered perfectly.
With the additional functionality gitea will be an ideal tool for
sharing resources with fellow software engineers working in the elixir
ecosystem. Allowing the git forge to be used without needing to install
any software.
## Feature Description
This issue requests the .livemd extension to be added as a Markdown
language extension.
- `.livemd` is the extension of Livebook which is an Elixir version of
Jupyter Notebook.
- `.livemd` is` a subset of Markdown.
This would require the .livemd to be recognized as a markdown file. The
Goldmark the markdown parser should handle the parsing and rendering
automatically.
Here is the corresponding commit for GitHub linguist:
https://github.com/github/linguist/pull/5672
Here is a sample page of a livemd file:
https://github.com/github/linguist/blob/master/samples/Markdown/livebook.livemd
## Screenshots
The first screenshot shows how github shows the sample .livemd in the
browser.
The second screenshot shows how mermaid js, renders my development
notebook and its corresponding ER Diagram. The source code can be found
here:
79615f7428/termiNotes.livemd
## Testing
I just changed the file extension from `.livemd`to `.md`and the document
already renders perfectly on codeberg. Check you can it out
[here](https://codeberg.org/lgh/Termi/src/branch/livemd2md/termiNotes.md)
---------
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
After #24317 this function is only used in one place where it is not
needed. I confirmed the timestamp still renders correctly
Signed-off-by: Yarden Shoham <git@yardenshoham.com>
This adds a API for getting License templates. This tries to be as close
to the [GitHub
API](https://docs.github.com/en/rest/licenses?apiVersion=2022-11-28) as
possible, but Gitea does not support all features that GitHub has. I
think they should been added, but this out f the scope of this PR. You
should merge #23006 before this PR for security reasons.
Enable [forbidigo](https://github.com/ashanbrown/forbidigo) linter which
forbids print statements. Will check how to integrate this with the
smallest impact possible, so a few `nolint` comments will likely be
required. Plan is to just go through the issues and either:
- Remove the print if it is nonsensical
- Add a `//nolint` directive if it makes sense
I don't plan on investigating the individual issues any further.
<details>
<summary>Initial Lint Results</summary>
```
modules/log/event.go:348:6: use of `fmt.Println` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Println(err)
^
modules/log/event.go:382:6: use of `fmt.Println` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Println(err)
^
modules/queue/unique_queue_disk_channel_test.go:20:2: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("TempDir %s\n", tmpDir)
^
contrib/backport/backport.go:168:2: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("* Backporting %s to %s as %s\n", pr, localReleaseBranch, backportBranch)
^
contrib/backport/backport.go:216:4: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("* Navigate to %s to open PR\n", url)
^
contrib/backport/backport.go:223:2: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("* `xdg-open %s`\n", url)
^
contrib/backport/backport.go:233:2: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("* `git push -u %s %s`\n", remote, backportBranch)
^
contrib/backport/backport.go:243:2: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("* Amending commit to prepend `Backport #%s` to body\n", pr)
^
contrib/backport/backport.go:272:3: use of `fmt.Println` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Println("* Attempting git cherry-pick --continue")
^
contrib/backport/backport.go:281:2: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("* Attempting git cherry-pick %s\n", sha)
^
contrib/backport/backport.go:297:2: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("* Current branch is %s\n", currentBranch)
^
contrib/backport/backport.go:299:3: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("* Current branch is %s - not checking out\n", currentBranch)
^
contrib/backport/backport.go:304:3: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("* Branch %s already exists. Checking it out...\n", backportBranch)
^
contrib/backport/backport.go:308:2: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("* `git checkout -b %s %s`\n", backportBranch, releaseBranch)
^
contrib/backport/backport.go:313:2: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("* `git fetch %s main`\n", remote)
^
contrib/backport/backport.go:316:3: use of `fmt.Println` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Println(string(out))
^
contrib/backport/backport.go:319:2: use of `fmt.Println` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Println(string(out))
^
contrib/backport/backport.go:321:2: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("* `git fetch %s %s`\n", remote, releaseBranch)
^
contrib/backport/backport.go:324:3: use of `fmt.Println` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Println(string(out))
^
contrib/backport/backport.go:327:2: use of `fmt.Println` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Println(string(out))
^
models/unittest/fixtures.go:50:3: use of `fmt.Println` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Println("Unsupported RDBMS for integration tests")
^
models/unittest/fixtures.go:89:3: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("LoadFixtures failed after retries: %v\n", err)
^
models/unittest/fixtures.go:110:4: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("Failed to generate sequence update: %v\n", err)
^
models/unittest/fixtures.go:117:6: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("Failed to update sequence: %s Error: %v\n", value, err)
^
models/migrations/base/tests.go:118:3: use of `fmt.Println` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Println("Environment variable $GITEA_ROOT not set")
^
models/migrations/base/tests.go:127:3: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("Could not find gitea binary at %s\n", setting.AppPath)
^
models/migrations/base/tests.go:134:3: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("Environment variable $GITEA_CONF not set - defaulting to %s\n", giteaConf)
^
models/migrations/base/tests.go:145:3: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("Unable to create temporary data path %v\n", err)
^
models/migrations/base/tests.go:154:3: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("Unable to InitFull: %v\n", err)
^
models/migrations/v1_11/v112.go:34:5: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("Error: %v", err)
^
contrib/fixtures/fixture_generation.go:36:3: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("CreateTestEngine: %+v", err)
^
contrib/fixtures/fixture_generation.go:40:3: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("PrepareTestDatabase: %+v\n", err)
^
contrib/fixtures/fixture_generation.go:46:5: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("generate '%s': %+v\n", r, err)
^
contrib/fixtures/fixture_generation.go:53:5: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("generate '%s': %+v\n", g.name, err)
^
contrib/fixtures/fixture_generation.go:71:4: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("%s created.\n", path)
^
services/gitdiff/gitdiff_test.go:543:2: use of `println` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
println(result)
^
services/gitdiff/gitdiff_test.go:560:2: use of `println` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
println(result)
^
services/gitdiff/gitdiff_test.go:577:2: use of `println` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
println(result)
^
modules/web/routing/logger_manager.go:34:2: use of `print` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
print Printer
^
modules/doctor/paths.go:109:3: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("Warning: can't remove temporary file: '%s'\n", tmpFile.Name())
^
tests/test_utils.go:33:2: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf(format+"\n", args...)
^
tests/test_utils.go:61:3: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("Environment variable $GITEA_CONF not set, use default: %s\n", giteaConf)
^
cmd/actions.go:54:9: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
_, _ = fmt.Printf("%s\n", respText)
^
cmd/admin_user_change_password.go:74:2: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("%s's password has been successfully updated!\n", user.Name)
^
cmd/admin_user_create.go:109:3: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("generated random password is '%s'\n", password)
^
cmd/admin_user_create.go:164:3: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("Access token was successfully created... %s\n", t.Token)
^
cmd/admin_user_create.go:167:2: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("New user '%s' has been successfully created!\n", username)
^
cmd/admin_user_generate_access_token.go:74:3: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("%s\n", t.Token)
^
cmd/admin_user_generate_access_token.go:76:3: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("Access token was successfully created: %s\n", t.Token)
^
cmd/admin_user_must_change_password.go:56:2: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("Updated %d users setting MustChangePassword to %t\n", n, mustChangePassword)
^
cmd/convert.go:44:3: use of `fmt.Println` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Println("Converted successfully, please confirm your database's character set is now utf8mb4")
^
cmd/convert.go:50:3: use of `fmt.Println` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Println("Converted successfully, please confirm your database's all columns character is NVARCHAR now")
^
cmd/convert.go:52:3: use of `fmt.Println` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Println("This command can only be used with a MySQL or MSSQL database")
^
cmd/doctor.go:104:3: use of `fmt.Println` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Println(err)
^
cmd/doctor.go:105:3: use of `fmt.Println` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Println("Check if you are using the right config file. You can use a --config directive to specify one.")
^
cmd/doctor.go:243:3: use of `fmt.Println` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Println(err)
^
cmd/embedded.go:154:3: use of `fmt.Println` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Println(a.path)
^
cmd/embedded.go:198:3: use of `fmt.Println` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Println("Using app.ini at", setting.CustomConf)
^
cmd/embedded.go:217:2: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("Extracting to %s:\n", destdir)
^
cmd/embedded.go:253:3: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("%s already exists; skipped.\n", dest)
^
cmd/embedded.go:275:2: use of `fmt.Println` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Println(dest)
^
cmd/generate.go:63:2: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("%s", internalToken)
^
cmd/generate.go:66:3: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("\n")
^
cmd/generate.go:78:2: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("%s", JWTSecretBase64)
^
cmd/generate.go:81:3: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("\n")
^
cmd/generate.go:93:2: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("%s", secretKey)
^
cmd/generate.go:96:3: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("\n")
^
cmd/keys.go:74:2: use of `fmt.Println` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Println(strings.TrimSpace(authorizedString))
^
cmd/mailer.go:32:4: use of `fmt.Print` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Print("warning: Content is empty")
^
cmd/mailer.go:35:3: use of `fmt.Print` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Print("Proceed with sending email? [Y/n] ")
^
cmd/mailer.go:40:4: use of `fmt.Println` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Println("The mail was not sent")
^
cmd/mailer.go:49:9: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
_, _ = fmt.Printf("Sent %s email(s) to all users\n", respText)
^
cmd/serv.go:147:3: use of `println` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
println("Gitea: SSH has been disabled")
^
cmd/serv.go:153:4: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("error showing subcommand help: %v\n", err)
^
cmd/serv.go:175:4: use of `println` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
println("Hi there! You've successfully authenticated with the deploy key named " + key.Name + ", but Gitea does not provide shell access.")
^
cmd/serv.go:177:4: use of `println` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
println("Hi there! You've successfully authenticated with the principal " + key.Content + ", but Gitea does not provide shell access.")
^
cmd/serv.go:179:4: use of `println` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
println("Hi there, " + user.Name + "! You've successfully authenticated with the key named " + key.Name + ", but Gitea does not provide shell access.")
^
cmd/serv.go:181:3: use of `println` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
println("If this is unexpected, please log in with password and setup Gitea under another user.")
^
cmd/serv.go:196:5: use of `fmt.Print` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Print(`{"type":"gitea","version":1}`)
^
tests/e2e/e2e_test.go:54:3: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("Error initializing test database: %v\n", err)
^
tests/e2e/e2e_test.go:63:3: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("util.RemoveAll: %v\n", err)
^
tests/e2e/e2e_test.go:67:3: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("Unable to remove repo indexer: %v\n", err)
^
tests/e2e/e2e_test.go:109:6: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("%v", stdout.String())
^
tests/e2e/e2e_test.go:110:6: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("%v", stderr.String())
^
tests/e2e/e2e_test.go:113:6: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("%v", stdout.String())
^
tests/integration/integration_test.go:124:3: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("Error initializing test database: %v\n", err)
^
tests/integration/integration_test.go:135:3: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("util.RemoveAll: %v\n", err)
^
tests/integration/integration_test.go:139:3: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("Unable to remove repo indexer: %v\n", err)
^
tests/integration/repo_test.go:357:4: use of `fmt.Printf` forbidden by pattern `^(fmt\.Print(|f|ln)|print|println)$` (forbidigo)
fmt.Printf("%s", resp.Body)
^
```
</details>
---------
Co-authored-by: Giteabot <teabot@gitea.io>
This refactors the `shared/datetime/short|long|full` templates into a
template helper function, which allows us to render absolute date times
within translatable phrases.
- Follows #23988
- The first attempt was in #24055
- This should help #22664
Changes:
1. Added the `DateTime` template helper that replaces the
`shared/datetime/short|long|full` templates
2. Used find-and-replace with varying regexes to replace the templates
from step 1 (for example, `\{\{template "shared/datetime/(\S+) \(dict
"Datetime" ([^"]+) "Fallback" ([^\)]+\)?) ?\)?\}\}` -> `{{DateTime "$1
$2 $3}}`)
3. Used the new `DateTime` helper in the issue due date timestamp
rendering
# Before
# After
---------
Signed-off-by: Yarden Shoham <git@yardenshoham.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
The `SHOW_FOOTER_BRANDING` came from year 2015, and it seems nobody ever
uses it. It only shows an GitHub icon which seems unrelated to Gitea, it
doesn't do what document says. So, remove it.
## ⚠️ Breaking
Users can now remove the key `[other].SHOW_FOOTER_BRANDING` from their
app.ini.
- Update all tool dependencies to latest tag
- Remove unused errcheck, it is part of golangci-lint
- Include main.go in air
- Enable wastedassign again now that it's
[generics-compatible](https://github.com/golangci/golangci-lint/pull/3689)
- Restructured lint targets to new `lint-*` namespace
Follow #23328
The improvements:
1. The `contains` functions are covered by tests
2. The inconsistent behavior of `containGeneric` is replaced by
`StringUtils.Contains` and `SliceUtils.Contains`
3. In the future we can move more help functions into XxxUtils to
simplify the `helper.go` and reduce unnecessary global functions.
FAQ:
1. Why it's called `StringUtils.Contains` but not `strings.Contains`
like Golang?
Because our `StringUtils` is not Golang's `strings` package. There will
be our own string functions.
---------
Co-authored-by: silverwind <me@silverwind.io>
Before, there was a `log/buffer.go`, but that design is not general, and
it introduces a lot of irrelevant `Content() (string, error) ` and
`return "", fmt.Errorf("not supported")` .
And the old `log/buffer.go` is difficult to use, developers have to
write a lot of `Contains` and `Sleep` code.
The new `LogChecker` is designed to be a general approach to help to
assert some messages appearing or not appearing in logs.
Users can now upload `webp` images.
Browsers supporting webp images then display this as the avatar of this
user (every major browser except IE).
---------
Co-authored-by: silverwind <me@silverwind.io>
The old code is unnecessarily complex, and has many misuses.
Old code "wraps" a lot, wrap wrap wrap, it's difficult to understand
which kind of handler is used.
The new code uses a general approach, we do not need to write all kinds
of handlers into the "wrapper", do not need to wrap them again and
again.
New code, there are only 2 concepts:
1. HandlerProvider: `func (h any) (handlerProvider func (next)
http.Handler)`, it can be used as middleware
2. Use HandlerProvider to get the final HandlerFunc, and use it for
`r.Get()`
And we can decouple the route package from context package (see the
TODO).
# FAQ
## Is `reflect` safe?
Yes, all handlers are checked during startup, see the `preCheckHandler`
comment. If any handler is wrong, developers could know it in the first
time.
## Does `reflect` affect performance?
No. https://github.com/go-gitea/gitea/pull/24080#discussion_r1164825901
1. This reflect code only runs for each web handler call, handler is far
more slower: 10ms-50ms
2. The reflect is pretty fast (comparing to other code): 0.000265ms
3. XORM has more reflect operations already
# Background
Golang template is not friendly for large projects, and Golang template
team is quite slow, related:
* `https://github.com/golang/go/issues/54450`
Without upstream support, we can also have our solution to make HTML
template functions support context.
It helps a lot, the above Golang template issue `#54450` explains a lot:
1. It makes `{{Locale.Tr}}` could be used in any template, without
passing unclear `(dict "root" . )` anymore.
2. More and more functions need `context`, like `avatar`, etc, we do not
need to do `(dict "Context" $.Context)` anymore.
3. Many request-related functions could be shared by parent&children
templates, like "user setting" / "system setting"
See the test `TestScopedTemplateSetFuncMap`, one template set, two
`Execute` calls with different `CtxFunc`.
# The Solution
Instead of waiting for upstream, this PR re-uses the escaped HTML
template trees, use `AddParseTree` to add related templates/trees to a
new template instance, then the new template instance can have its own
FuncMap , the function calls in the template trees will always use the
new template's FuncMap.
`template.New` / `template.AddParseTree` / `adding-FuncMap` are all
quite fast, so the performance is not affected.
The details:
1. Make a new `html/template/Template` for `all` templates
2. Add template code to the `all` template
3. Freeze the `all` template, reset its exec func map, it shouldn't
execute any template.
4. When a router wants to render a template by its `name`
1. Find the `name` in `all`
2. Find all its related sub templates
3. Escape all related templates (just like what the html template
package does)
4. Add the escaped parse-trees of related templates into a new (scoped)
`text/template/Template`
5. Add context-related func map into the new (scoped) text template
6. Execute the new (scoped) text template
7. To improve performance, the escaped templates are cached to `template
sets`
# FAQ
## There is a `unsafe` call, is this PR unsafe?
This PR is safe. Golang has strict language definition, it's safe to do
so: https://pkg.go.dev/unsafe#Pointer (1) Conversion of a *T1 to Pointer
to *T2
## What if Golang template supports such feature in the future?
The public structs/interfaces/functions introduced by this PR is quite
simple, the code of `HTMLRender` is not changed too much. It's very easy
to switch to the official mechanism if there would be one.
## Does this PR change the template execution behavior?
No, see the tests (welcome to design more tests if it's necessary)
---------
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: Giteabot <teabot@gitea.io>
Close#7570
1. Clearly define the wiki path behaviors, see
`services/wiki/wiki_path.go` and tests
2. Keep compatibility with old contents
3. Allow to use dashes in titles, eg: "2000-01-02 Meeting record"
4. Add a "Pages" link in the dropdown, otherwise users can't go to the
Pages page easily.
5. Add a "View original git file" link in the Pages list, even if some
file names are broken, users still have a chance to edit or remove it,
without cloning the wiki repo to local.
6. Fix 500 error when the name contains prefix spaces.
This PR also introduces the ability to support sub-directories, but it
can't be done at the moment due to there are a lot of legacy wiki data,
which use "%2F" in file names.
Co-authored-by: Giteabot <teabot@gitea.io>
The old code has a lot of technical debts, eg: `repo/wiki/view.tmpl` /
`Iterate`
This PR improves the Wiki TOC display and improves the code.
---------
Co-authored-by: delvh <dev.lh@web.de>
Close#24062
At the beginning, I just wanted to fix the warning mentioned by #24062
But, the cookie code really doesn't look good to me, so clean up them.
Complete the TODO on `SetCookie`:
> TODO: Copied from gitea.com/macaron/macaron and should be improved
after macaron removed.
The default access log format has been unnecessarily escaped, leading to
spurious backslashes appearing in log lines.
Additionally, the `RemoteAddr` field includes the port, which breaks
most log parsers attempting to process it. I've added a call to
`net.SplitHostPort()` attempting to isolate the address alone, with a
fallback to the original address if it errs.
Signed-off-by: Gary Moon <gary@garymoon.net>
The idea is to use a Layered Asset File-system (modules/assetfs/layered.go)
For example: when there are 2 layers: "custom", "builtin", when access
to asset "my/page.tmpl", the Layered Asset File-system will first try to
use "custom" assets, if not found, then use "builtin" assets.
This approach will hugely simplify a lot of code, make them testable.
Other changes:
* Simplify the AssetsHandlerFunc code
* Simplify the `gitea embedded` sub-command code
---------
Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Without this patch, the setting SSH.StartBuiltinServer decides whether
the native (Go) implementation is used rather than calling 'ssh-keygen'.
It's possible for 'using ssh-keygen' and 'using the built-in server' to
be independent.
In fact, the gitea rootless container doesn't ship ssh-keygen and can be
configured to use the host's SSH server - which will cause the public
key parsing mechanism to break.
This commit changes the decision to be based on SSH.KeygenPath instead.
Any existing configurations with a custom KeygenPath set will continue
to function. The new default value of '' selects the native version. The
downside of this approach is that anyone who has relying on plain
'ssh-keygen' to have special properties will now be using the native
version instead.
I assume the exec-variant is only there because /x/crypto/ssh didn't
support ssh-ed25519 until 2016. I don't see any other reason for using
it so it might be an acceptable risk.
Fixes#23363
EDIT: this message was garbled when I tried to get the commit
description back in.. Trying to reconstruct it:
## ⚠️ BREAKING ⚠️ Users who don't have SSH.KeygenPath
explicitly set and rely on the ssh-keygen binary need to set
SSH.KeygenPath to 'ssh-keygen' in order to be able to continue using it
for public key parsing.
There was something else but I can't remember at the moment.
EDIT2: It was about `make test` and `make lint`. Can't get them to run.
To reproduce the issue, I installed `golang` in `docker.io/node:16` and
got:
```
...
go: mvdan.cc/xurls/v2@v2.4.0: unknown revision mvdan.cc/xurls/v2.4.0
go: gotest.tools/v3@v3.4.0: unknown revision gotest.tools/v3.4.0
...
go: gotest.tools/v3@v3.0.3: unknown revision gotest.tools/v3.0.3
...
go: error loading module requirements
```
Signed-off-by: Leon M. Busch-George <leon@georgemail.eu>
Close#24002
Two problems before:
1. The `log.Fatal` is missing after these `wrapFatal` calls, so the
error is not shown to users.
2. `GetTemplateAssetNames` has different behaviors for local files and
builtin assets, for builtin assets, it also returns directories, so we
need to check the extension again.
I have tested with `TAGS="bindata sqlite sqlite_unlock_notify" make
build && ./gitea` , it works well now. Before, the server responds
internal server error (because it doesn't complete the template parsing)
None of the features of `unrolled/render` package is used.
The Golang builtin "html/template" just works well. Then we can improve
our HTML render to resolve the "$.root.locale.Tr" problem as much as
possible.
Next step: we can have a template render pool (by Clone), then we can
inject global functions with dynamic context to every `Execute` calls.
Then we can use `{{Locale.Tr ....}}` directly in all templates , no need
to pass the `$.root.locale` again and again.
One of the steps in #23328
Before there were 3 different but similar functions: dict/Dict/mergeinto
The code was just copied & pasted, no test.
This PR defines a new stable `dict` function, it covers all the 3 old
functions behaviors, only +160 -171
Future developers do not need to think about or guess the different dict
functions, just use one: `dict`
Why use `dict` but not `Dict`? Because there are far more `dict` than
`Dict` in code already ......
One of the proposals in #23328
This PR introduces a simple expression calculator
(templates/eval/eval.go), it can do basic expression calculations.
Many untested template helper functions like `Mul` `Add` can be replaced
by this new approach.
Then these `Add` / `Mul` / `percentage` / `Subtract` / `DiffStatsWidth`
could all use this `Eval`.
And it provides enhancements for Golang templates, and improves
readability.
Some examples:
----
* Before: `{{Add (Mul $glyph.Row 12) 12}}`
* After: `{{Eval $glyph.Row "*" 12 "+" 12}}`
----
* Before: `{{if lt (Add $i 1) (len $.Topics)}}`
* After: `{{if Eval $i "+" 1 "<" (len $.Topics)}}`
## FAQ
### Why not use an existing expression package?
We need a highly customized expression engine:
* do the calculation on the fly, without pre-compiling
* deal with int/int64/float64 types, to make the result could be used in
Golang template.
* make the syntax could be used in the Golang template directly
* do not introduce too much complex or strange syntax, we just need a
simple calculator.
* it needs to strictly follow Golang template's behavior, for example,
Golang template treats all non-zero values as truth, but many 3rd
packages don't do so.
### What's the benefit?
* Developers don't need to add more `Add`/`Mul`/`Sub`-like functions,
they were getting more and more.
Now, only one `Eval` is enough for all cases.
* The new code reads better than old `{{Add (Mul $glyph.Row 12) 12}}`,
the old one isn't familiar to most procedural programming developers
(eg, the Golang expression syntax).
* The `Eval` is fully covered by tests, many old `Add`/`Mul`-like
functions were never tested.
### The performance?
It doesn't use `reflect`, it doesn't need to parse or compile when used
in Golang template, the performance is as fast as native Go template.
### Is it too complex? Could it be unstable?
The expression calculator program is a common homework for computer
science students, and it's widely used as a teaching and practicing
purpose for developers. The algorithm is pretty well-known.
The behavior can be clearly defined, it is stable.
The first step of #23328
This PR cleans:
1. Dead function like `SubStr`
2. Unnecessary function like `UseHTTPS`, it should be provided by
`window.origin`
3. Duplicate function like `IsShowFullName`, there was already a
`DeafultShowFullName`
I have searched these removed functions globally, no use in code.
Remove the misbehaving function and call
Repository.GetFilesChangedBetween instead.
Fixes#23919
---
~~_TODO_ test this~~ `Repository.getFilesChanged` seems to be only used
by Gitea Actions, but a similar function already exists
**Update** I tested this change and the issue is gone.
The _graceful_ should fail less when the `.editorconfig` file isn't
properly written, e.g. boolean values from YAML or unparseable numbers
(when a number is expected). As is... information is lost as the
_warning_ (a go-multierror.Error) is ignored. If anybody knows how to
send them to the UI as warning; any help is appreciated.
Closes#20694
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
At first, we have one unified team unit permission which is called
`Team.Authorize` in DB.
But since https://github.com/go-gitea/gitea/pull/17811, we allowed
different units to have different permission.
The old code is only designed for the old version. So after #17811, if
org users have write permission of other units, but have no permission
of packages, they can also get write permission of packages.
Co-authored-by: delvh <dev.lh@web.de>
Follow #23633 and #23240Close#23814
Now we almost have a complete test set for Gitea's LocalStore.
This PR is still a quick fix for the legacy locale system (see the
TODOs), to resolve the problems fundamentally, it needs more work in the
future.
Follow #21962
After I eat my own dogfood, I would say that
ONLY_SHOW_RELEVANT_REPOS=false is necessary for many private/enterprise
instances, because many private repositories do not have
"description/topic", users just want to search by their names.
This PR also adds `PageIsExploreRepositories` check, to make code more
strict, because the `search` template is shared for different purpose.
And during the test, I found a bug that the "Search" button didn't
respect the "relevant" parameter, so this PR fixes the bug by the way
together.
I think this PR needs to be backported.
# Why this PR comes
At first, I'd like to help users like #23636 (there are a lot)
The unclear "Internal Server Error" is quite anonying, scare users,
frustrate contributors, nobody knows what happens.
So, it's always good to provide meaningful messages to end users (of
course, do not leak sensitive information).
When I started working on the "response message to end users", I found
that the related code has a lot of technical debt. A lot of copy&paste
code, unclear fields and usages.
So I think it's good to make everything clear.
# Tech Backgrounds
Gitea has many sub-commands, some are used by admins, some are used by
SSH servers or Git Hooks. Many sub-commands use "internal API" to
communicate with Gitea web server.
Before, Gitea server always use `StatusCode + Json "err" field` to
return messages.
* The CLI sub-commands: they expect to show all error related messages
to site admin
* The Serv/Hook sub-commands (for git clients): they could only show
safe messages to end users, the error log could only be recorded by
"SSHLog" to Gitea web server.
In the old design, it assumes that:
* If the StatusCode is 500 (in some functions), then the "err" field is
error log, shouldn't be exposed to git client.
* If the StatusCode is 40x, then the "err" field could be exposed. And
some functions always read the "err" no matter what the StatusCode is.
The old code is not strict, and it's difficult to distinguish the
messages clearly and then output them correctly.
# This PR
To help to remove duplicate code and make everything clear, this PR
introduces `ResponseExtra` and `requestJSONResp`.
* `ResponseExtra` is a struct which contains "extra" information of a
internal API response, including StatusCode, UserMsg, Error
* `requestJSONResp` is a generic function which can be used for all
cases to help to simplify the calls.
* Remove all `map["err"]`, always use `private.Response{Err}` to
construct error messages.
* User messages and error messages are separated clearly, the `fail` and
`handleCliResponseExtra` will output correct messages.
* Replace all `Internal Server Error` messages with meaningful (still
safe) messages.
This PR saves more than 300 lines, while makes the git client messages
more clear.
Many gitea-serv/git-hook related essential functions are covered by
tests.
---------
Co-authored-by: delvh <dev.lh@web.de>
Closes#20955
This PR adds the possibility to disable blank Issues, when the Repo has
templates. This can be done by creating the file
`.gitea/issue_config.yaml` with the content `blank_issues_enabled` in
the Repo.
