finn/forgejo
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Graceful Queues: Issue Indexing and Tasks (#9363)

Browse source 
* Queue: Add generic graceful queues with settings

* Queue & Setting: Add worker pool implementation

* Queue: Add worker settings

* Queue: Make resizing worker pools

* Queue: Add name variable to queues

* Queue: Add monitoring

* Queue: Improve logging

* Issues: Gracefulise the issues indexer

Remove the old now unused specific queues

* Task: Move to generic queue and gracefulise

* Issues: Standardise the issues indexer queue settings

* Fix test

* Queue: Allow Redis to connect to unix

* Prevent deadlock during early shutdown of issue indexer

* Add MaxWorker settings to queues

* Merge branch 'master' into graceful-queues

* Update modules/indexer/issues/indexer.go

Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>

* Update modules/indexer/issues/indexer.go

Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>

* Update modules/queue/queue_channel.go

Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>

* Update modules/queue/queue_disk.go

* Update modules/queue/queue_disk_channel.go

Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>

* Rename queue.Description to queue.ManagedQueue as per @guillep2k

* Cancel pool workers when removed

* Remove dependency on queue from setting

* Update modules/queue/queue_redis.go

Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>

* As per @guillep2k add mutex locks on shutdown/terminate

* move unlocking out of setInternal

* Add warning if number of workers < 0

* Small changes as per @guillep2k

* No redis host specified not found

* Clean up documentation for queues

* Update docs/content/doc/advanced/config-cheat-sheet.en-us.md

* Update modules/indexer/issues/indexer_test.go

* Ensure that persistable channel queue is added to manager

* Rename QUEUE_NAME REDIS_QUEUE_NAME

* Revert "Rename QUEUE_NAME REDIS_QUEUE_NAME"

This reverts commit 1f83b4fc9b9dabda186257b38c265fe7012f90df.

Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: techknowlogick <matti@mdranta.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
This commit is contained in:
zeripath 2020-01-07 11:23:09 +00:00 • committed by Antoine GIRARD
parent f71e1c8e79
commit 62eb1b0f25
35 changed files with 2875 additions and 647 deletions
Download patch file Download diff file Expand all files Collapse all files

50
options/locale/locale_en-US.ini
View file

@ -1410,7 +1410,7 @@ settings.protect_check_status_contexts_list = Status checks found in the last we
settings.protect_required_approvals = Required approvals:
settings.protect_required_approvals_desc = Allow only to merge pull request with enough positive reviews.
settings.protect_approvals_whitelist_enabled = Restrict approvals to whitelisted users or teams
settings.protect_approvals_whitelist_enabled_desc = Only reviews from whitelisted users or teams will count to the required approvals. Without approval whitelist, reviews from anyone with write access count to the required approvals.
settings.protect_approvals_whitelist_enabled_desc = Only reviews from whitelisted users or teams will count to the required approvals. Without approval whitelist, reviews from anyone with write access count to the required approvals.
settings.protect_approvals_whitelist_users = Whitelisted reviewers:
settings.protect_approvals_whitelist_teams = Whitelisted teams for reviews:
settings.add_protected_branch = Enable protection
@ -2026,6 +2026,54 @@ monitor.execute_time = Execution Time
monitor.process.cancel = Cancel process
monitor.process.cancel_desc = Cancelling a process may cause data loss
monitor.process.cancel_notices = Cancel: <strong>%s</strong>?
monitor.queues = Queues
monitor.queue = Queue: %s
monitor.queue.name = Name
monitor.queue.type = Type
monitor.queue.exemplar = Exemplar Type
monitor.queue.numberworkers = Number of Workers
monitor.queue.maxnumberworkers = Max Number of Workers
monitor.queue.review = Review Config
monitor.queue.review_add = Review/Add Workers
monitor.queue.configuration = Initial Configuration
monitor.queue.nopool.title = No Worker Pool
monitor.queue.nopool.desc = This queue wraps other queues and does not itself have a worker pool.
monitor.queue.wrapped.desc = A wrapped queue wraps a slow starting queue, buffering queued requests in a channel. It does not have a worker pool itself.
monitor.queue.persistable-channel.desc = A persistable-channel wraps two queues, a channel queue that has its own worker pool and a level queue for persisted requests from previous shutdowns. It does not have a worker pool itself.
monitor.queue.pool.timeout = Timeout
monitor.queue.pool.addworkers.title = Add Workers
monitor.queue.pool.addworkers.submit = Add Workers
monitor.queue.pool.addworkers.desc = Add Workers to this pool with or without a timeout. If you set a timeout these workers will be removed from the pool after the timeout has lapsed.
monitor.queue.pool.addworkers.numberworkers.placeholder = Number of Workers
monitor.queue.pool.addworkers.timeout.placeholder = Set to 0 for no timeout
monitor.queue.pool.addworkers.mustnumbergreaterzero = Number of Workers to add must be greater than zero
monitor.queue.pool.addworkers.musttimeoutduration = Timeout must be a golang duration eg. 5m or be 0
monitor.queue.settings.title = Pool Settings
monitor.queue.settings.desc = Pools dynamically grow with a boost in response to their worker queue blocking. These changes will not affect current worker groups.
monitor.queue.settings.timeout = Boost Timeout
monitor.queue.settings.timeout.placeholder = Currently %[1]v
monitor.queue.settings.timeout.error = Timeout must be a golang duration eg. 5m or be 0
monitor.queue.settings.numberworkers = Boost Number of Workers
monitor.queue.settings.numberworkers.placeholder = Currently %[1]d
monitor.queue.settings.numberworkers.error = Number of Workers to add must be greater than or equal to zero
monitor.queue.settings.maxnumberworkers = Max Number of workers
monitor.queue.settings.maxnumberworkers.placeholder = Currently %[1]d
monitor.queue.settings.maxnumberworkers.error = Max number of workers must be a number
monitor.queue.settings.submit = Update Settings
monitor.queue.settings.changed = Settings Updated
monitor.queue.settings.blocktimeout = Current Block Timeout
monitor.queue.settings.blocktimeout.value = %[1]v
monitor.queue.pool.none = This queue does not have a Pool
monitor.queue.pool.added = Worker Group Added
monitor.queue.pool.max_changed = Maximum number of workers changed
monitor.queue.pool.workers.title = Active Worker Groups
monitor.queue.pool.workers.none = No worker groups.
monitor.queue.pool.cancel = Shutdown Worker Group
monitor.queue.pool.cancelling = Worker Group shutting down
monitor.queue.pool.cancel_notices = Shutdown this group of %s workers?
monitor.queue.pool.cancel_desc = Leaving a queue without any worker groups may cause requests to block indefinitely.
notices.system_notice_list = System Notices
notices.view_detail_header = View Notice Details