-
Notifications
You must be signed in to change notification settings - Fork 990
Labels
configItems related to configuring augur settings/state to change how it behavesItems related to configuring augur settings/state to change how it behavesgood first issuesGood issues for a newcomer to tackleGood issues for a newcomer to tackle
Description
We still have several periodic tasks configured in celery with fixed time intervals. These should be pulling their intervals from the config just like the materialized view task
augur/augur/tasks/init/celery_app.py
Lines 231 to 260 in 6b48ab6
| config = AugurConfig(logger, session) | |
| collection_interval = config.get_value('Tasks', 'collection_interval') | |
| logger.info(f"Scheduling collection every {collection_interval/60} minutes") | |
| sender.add_periodic_task(collection_interval, augur_collection_monitor.s()) | |
| #Do longer tasks less often | |
| logger.info(f"Scheduling data analysis every 30 days") | |
| thirty_days_in_seconds = 30*24*60*60 | |
| sender.add_periodic_task(thirty_days_in_seconds, non_repo_domain_tasks.s()) | |
| mat_views_interval = int(config.get_value('Celery', 'refresh_materialized_views_interval_in_days')) | |
| if mat_views_interval > 0: | |
| logger.info(f"Scheduling refresh materialized view every night at 1am CDT") | |
| sender.add_periodic_task(datetime.timedelta(days=mat_views_interval), refresh_materialized_views.s()) | |
| else: | |
| logger.info(f"Refresh materialized view task is disabled.") | |
| # logger.info(f"Scheduling update of collection weights on midnight each day") | |
| # sender.add_periodic_task(crontab(hour=0, minute=0),augur_collection_update_weights.s()) | |
| logger.info(f"Setting 404 repos to be marked for retry on midnight each day") | |
| sender.add_periodic_task(crontab(hour=0, minute=0),retry_errored_repos.s()) | |
| one_hour_in_seconds = 60*60 | |
| sender.add_periodic_task(one_hour_in_seconds, process_contributors.s()) | |
| one_day_in_seconds = 24*60*60 | |
| sender.add_periodic_task(one_day_in_seconds, create_collection_status_records.s()) |
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
configItems related to configuring augur settings/state to change how it behavesItems related to configuring augur settings/state to change how it behavesgood first issuesGood issues for a newcomer to tackleGood issues for a newcomer to tackle