CRON Engine

Lombi	#1 2008-08-21 01:41
Team Thanked: 1 time	I think we need to make a cron engine for Seditio, in the same way that Invision Power Board has it's Task Manager and vBulletin has it's Smart Tasks. basically this system would take care of all "cron" things, but would of course only be aplicable if there's more than 0 people on the site (if noone comes, then nothing gets triggered, like the two forum engines). When the first person comes, the stuff that's been waiting gets run. Your thoughts? <a href="http://www.domenlo.com">Surreal Art</a>

Lombi

#1 2008-08-21 01:41

I think we need to make a cron engine for Seditio, in the same way that Invision Power Board has it's Task Manager and vBulletin has it's Smart Tasks.

basically this system would take care of all "cron" things, but would of course only be aplicable if there's more than 0 people on the site (if noone comes, then nothing gets triggered, like the two forum engines). When the first person comes, the stuff that's been waiting gets run.

Your thoughts?

<a href="http://www.domenlo.com">Surreal Art</a>

Trustmaster	#2 2008-08-21 04:30
Administrators Thanked: 263 times	There are actually plugins that act like that. Mostly for Admin part, but they could use global hook instead. But there are a few pitfalls with this approach: 1. Performance loss due to overhead of calling task manager each request (at least a simple check, but still each request). 2. Interoperability and race conditions. This should be programmed in a "thead-safe" fashion. May the Source be with you!

Trustmaster

#2 2008-08-21 04:30

Administrators
Thanked: 263 times

There are actually plugins that act like that. Mostly for Admin part, but they could use global hook instead. But there are a few pitfalls with this approach:
1. Performance loss due to overhead of calling task manager each request (at least a simple check, but still each request).
2. Interoperability and race conditions. This should be programmed in a "thead-safe" fashion.

May the Source be with you!

Lombi	#3 2008-08-21 20:57
Team Thanked: 1 time	Right. Or at least one cron to run them all (so you cron the task manager to run once per 5 minutes and have it run everything else) <a href="http://www.domenlo.com">Surreal Art</a>

Lombi	#4 2008-08-21 21:00
Team Thanked: 1 time	Also the admin hook might be extremely useless for some people that actually dont visit the admin front page every two days. That's why we did a cron cleaner instead of a admin hook cleaner. <a href="http://www.domenlo.com">Surreal Art</a>

Trustmaster	#5 2008-08-22 06:07
Administrators Thanked: 263 times	Running something like cron.php/tasks.php with real cron would be a better solution. May the Source be with you!

diablo	#6 2009-02-10 05:08
Members Thanked: 5 times	maybe we can use the cache system for this. "Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live."

Trustmaster	#7 2009-02-10 05:19
Administrators Thanked: 263 times	There's a high risk of getting into race condition on busy sites. May the Source be with you!

diablo	#8 2009-02-10 05:32
Members Thanked: 5 times	i dont know much about race condition but here is my idea we can use a second check for unique cron job name for instance we will run the job "Inactive User Cleaner" its unique name lets say "iuc" it will check the cache first if its expired it will attempt to add it to its "to-do list" since the job names gonna be unique it will give error when it attempts to add "iuc" to the list. so the at the 1st proc it will be added to list but at 2nd proc it will stop since it will fail adding it to the "list" so check cache() if expired { check list() if not in list { add to list* } else { die } } execute cron job list * since it wont be able to add list the function wont work further "Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live."
This post was edited by diablo (2009-02-10 05:40, 15 years ago)

diablo

#8 2009-02-10 05:32

Members
Thanked: 5 times

i dont know much about race condition but here is my idea
we can use a second check for unique cron job name
for instance we will run the job "Inactive User Cleaner" its unique name lets say "iuc"
it will check the cache first if its expired it will attempt to add it to its "to-do list" since the job names gonna be unique it will give error when it attempts to add "iuc" to the list. so the at the 1st proc it will be added to list but at 2nd proc it will stop since it will fail adding it to the "list"

so

check cache()
if expired
{
check list()
if not in list
{
add to list*
}
else
{
die
}

}
execute cron job list

* since it wont be able to add list the function wont work further

"Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live."

This post was edited by diablo (2009-02-10 05:40, 15 years ago)

Kilandor	#9 2009-02-10 14:41
Administrators Thanked: 9 times	While I don't see to much need for this I guess some may like it. TrackBot and Ticketbot, and other plugins i've created run on a cron like feature, they only execute every X time, all you have to do is create a config field to store the time. There is even plans for "hidden" config fields in the future to hide things like this. The "race condition" is just referring to, multiple things possibly executing at the same time, which conflict each other and causes problems. But maybe a better improvement to this feature would be some sort of timer feature for plugin hooks. There could be an extra field in hooks for a timer, so they only are included every so often, then when the script is loaded it can use a function like sed_hook_timer('plug', 'hook', 'newtime'); Then it won't be executed again untill that time. This would be the best way to do it, No fancy extra complete system for it. All it requires is a new column, and a slight change to hooks. And we can add an new function for easy setting. And we could even go so far as for the admin panel to create on plugin list, to reset timers for all plugins, and for each plugin specifically in the plugin page. Actually after my idea here, i like it alot. If there are no objections, i'll implement it into the core. As it will also cut back for some people on includes that they don't need. Cache system definatly should not be used for this, thats not its purpose in anyway

Kilandor

#9 2009-02-10 14:41

Administrators
Thanked: 9 times

While I don't see to much need for this I guess some may like it. TrackBot and Ticketbot, and other plugins i've created run on a cron like feature, they only execute every X time, all you have to do is create a config field to store the time. There is even plans for "hidden" config fields in the future to hide things like this.

The "race condition" is just referring to, multiple things possibly executing at the same time, which conflict each other and causes problems.

But maybe a better improvement to this feature would be some sort of timer feature for plugin hooks. There could be an extra field in hooks for a timer, so they only are included every so often, then when the script is loaded it can use a function like sed_hook_timer('plug', 'hook', 'newtime'); Then it won't be executed again untill that time.

This would be the best way to do it, No fancy extra complete system for it. All it requires is a new column, and a slight change to hooks. And we can add an new function for easy setting. And we could even go so far as for the admin panel to create on plugin list, to reset timers for all plugins, and for each plugin specifically in the plugin page.

Actually after my idea here, i like it alot. If there are no objections, i'll implement it into the core. As it will also cut back for some people on includes that they don't need.

Cache system definatly should not be used for this, thats not its purpose in anyway

Mynt	#10 2009-08-18 15:46
Donator Thanked: 1 time	Cron jobs, while great, can be a nuesance because of the lack of need. Not saying that recalculating forums posts, threads, and user syncs wouldn't be a great idea I don't see a large lackluster of mem leaks happening anywhere in which the synchronization would be pushed to far out of the loop. Unless you plan on recaching information into the database, I don't see a need for a cron yet. The best way to make a cron task though is to do it GDLIB style. I find this is the most effecient way. Attaching a simple internal counter to a blank gd_image for views. When the file gets poached and it generates a new one after xxx views, then run the cron. I have set many of those up and see that as being the most effecient since it waits til xxx db queries before it even thinks of running a cron. Pointless to run a crontab on a site that gets less than 600 hits a day with less than 300 posts a day, you know? "I only sleep when I know I won't miss a thing."

Mynt

#10 2009-08-18 15:46

Donator
Thanked: 1 time

Cron jobs, while great, can be a nuesance because of the lack of need. Not saying that recalculating forums posts, threads, and user syncs wouldn't be a great idea I don't see a large lackluster of mem leaks happening anywhere in which the synchronization would be pushed to far out of the loop. Unless you plan on recaching information into the database, I don't see a need for a cron yet.

The best way to make a cron task though is to do it GDLIB style. I find this is the most effecient way. Attaching a simple internal counter to a blank gd_image for views. When the file gets poached and it generates a new one after xxx views, then run the cron. I have set many of those up and see that as being the most effecient since it waits til xxx db queries before it even thinks of running a cron. Pointless to run a crontab on a site that gets less than 600 hits a day with less than 300 posts a day, you know?

"I only sleep when I know I won't miss a thing."

Trustmaster	#11 2009-08-18 17:33
Administrators Thanked: 263 times	That would work if you get less than 600 hits. If you get more than 60000 that would be better to use cron instead. Both would invoke same code actually, so a daemonizable scripts should be written first, then we provide several ways of daemonizing them. May the Source be with you!

Mynt	#12 2009-08-18 17:44
Donator Thanked: 1 time	# Trustmaster : That would work if you get less than 600 hits. If you get more than 60000 that would be better to use cron instead. Both would invoke same code actually, so a daemonizable scripts should be written first, then we provide several ways of daemonizing them. So the idea is to add one in place, with a feature to run it if wanted? Or would you have a check balance system to run it. Also, alot of people have a hard time just setting up a blank dB in MySQL, so how would you advise running a basic cron? (no offense to anyone). You could just create psuedo crons to just execute various things. Maybe to failsafe people using Windows based machines instead of *nix. http://www.bitfolge.de/pseudocron-en.html "I only sleep when I know I won't miss a thing."

Mynt

#12 2009-08-18 17:44

Donator
Thanked: 1 time

# Trustmaster : That would work if you get less than 600 hits. If you get more than 60000 that would be better to use cron instead. Both would invoke same code actually, so a daemonizable scripts should be written first, then we provide several ways of daemonizing them.

So the idea is to add one in place, with a feature to run it if wanted? Or would you have a check balance system to run it. Also, alot of people have a hard time just setting up a blank dB in MySQL, so how would you advise running a basic cron? (no offense to anyone).

You could just create psuedo crons to just execute various things. Maybe to failsafe people using Windows based machines instead of *nix.

http://www.bitfolge.de/pseudocron-en.html

"I only sleep when I know I won't miss a thing."

Kilandor	#13 2009-08-18 18:34
Administrators Thanked: 9 times	Nowadays creating a cron is as simple as choosing options from a few select boxes on cpanel and such. I can assure you that any sort of cron that works based off user's accessing a website is not good and does not always work correctly. It very easily can run multiple times, I know this for a fact of how we used to run my Ticket/Revision bot, it was using a hook, that was time based and often was executed multiple times, even implementing a file locking method it was still getting ran multiple times. So no such solution can work perfect, and duplicate exectuions will happen. As well as most typical users won't need a cron period anyways.

Kilandor

#13 2009-08-18 18:34

Administrators
Thanked: 9 times

Nowadays creating a cron is as simple as choosing options from a few select boxes on cpanel and such.

I can assure you that any sort of cron that works based off user's accessing a website is not good and does not always work correctly.

It very easily can run multiple times, I know this for a fact of how we used to run my Ticket/Revision bot, it was using a hook, that was time based and often was executed multiple times, even implementing a file locking method it was still getting ran multiple times. So no such solution can work perfect, and duplicate exectuions will happen.

As well as most typical users won't need a cron period anyways.

Mynt	#14 2009-08-19 06:47
Donator Thanked: 1 time	@Kilandor, I agree, which re-iterates my first statement of, I don't see a need for a crontab at all. Though, instead of a cron, maybe a simple UI that just says every month, HEY BACK UP, HEY RESYNCH, HEY DO THIS. "I only sleep when I know I won't miss a thing."