Sometimes task run before previous is not finished

[riksking]

Hi.
I have ONE chain with ONE task that run every 5 seconds.
But sometimes when task runs more then 5 seconds new task starts.
And this leads to errors in the execution of the procedure called in the task.

From timetable.execution_log:

"chain_id"	"task_id"	"txid"	"last_run"	"finished"	"pid"	"returncode"
4	5	3376626675	"2024-08-09 11:47:46.596801+03"	"2024-08-09 11:47:50.257091+03"	1329497618	0
4	5	3376624827	"2024-08-09 11:47:41.560742+03"	"2024-08-09 11:47:50.046199+03"	1329497618	0

From timetable.log:

"ts"	"pid"	"log_level"	"client_name"	"message"	"message_data"
"2024-08-09 11:47:50.257538+03"	1329497618	"INFO"	"pg_timetable"	"Chain executed successfully"	"{""txid"": 3376626675, ""chain"": 4}"
"2024-08-09 11:47:50.255304+03"	1329497618	"INFO"	"pg_timetable"	"Task executed successfully"	"{""task"": 5, ""txid"": 3376626675, ""chain"": 4}"
"2024-08-09 11:47:50.046528+03"	1329497618	"INFO"	"pg_timetable"	"Chain executed successfully"	"{""txid"": 3376624827, ""chain"": 4}"
"2024-08-09 11:47:50.044103+03"	1329497618	"INFO"	"pg_timetable"	"Task executed successfully"	"{""task"": 5, ""txid"": 3376624827, ""chain"": 4}"
"2024-08-09 11:47:46.589178+03"	1329497618	"INFO"	"pg_timetable"	"Starting task"	"{""task"": 5, ""txid"": 3376626675, ""chain"": 4}"
"2024-08-09 11:47:46.533963+03"	1329497618	"INFO"	"pg_timetable"	"Starting chain"	"{""chain"": 4}"
"2024-08-09 11:47:44.395817+03"	1329497618	"INFO"	"pg_timetable"	"Retrieve interval chains to run"	"{""count"": 3}"
"2024-08-09 11:47:44.393417+03"	1329497618	"INFO"	"pg_timetable"	"Retrieve scheduled chains to run"	"{""count"": 0}"
"2024-08-09 11:47:41.548341+03"	1329497618	"INFO"	"pg_timetable"	"Starting task"	"{""task"": 5, ""txid"": 3376624827, ""chain"": 4}"
"2024-08-09 11:47:41.533528+03"	1329497618	"INFO"	"pg_timetable"	"Starting chain"	"{""chain"": 4}"

Can you tell me how to avoid parallel task launches?

[pashagolub]

Hello. Would you please provide a definition of the chain?

Please set max_instances to limit the amount of instances that this chain may have running at the same time.

[pashagolub]

Is it REFRESH MATERIALIZED VIEW CONCURRENTLY?

[pashagolub]

CREATE OR REPLACE FUNCTION sleepy_func(text)
  RETURNS void LANGUAGE plpgsql AS
$BODY$ 
BEGIN 
   RAISE NOTICE 'Sleeping for 15 sec in %', $1;
   PERFORM pg_sleep_for('15 seconds');
   RAISE NOTICE 'Waking up in %', $1;
END; 
$BODY$;

SELECT timetable.add_job(
    job_name            => 'exclusive sleepy task every 5 sec',
    job_schedule        => '@every 5 seconds',
    job_command         => 'SELECT sleepy_func($1)',
    job_parameters      => '[ "Configuration EVERY 10sec" ]'::jsonb,
    job_kind            => 'SQL'::timetable.command_kind,
    job_max_instances   => 1,
    job_live            => TRUE,
    job_exclusive       => FALSE
) 

I tried this approach but cannot reproduce the error. Can you show the full timetable.execution_log including client_name. I suspect you have more than one parallel worker not interfering with each other

[riksking]

Is it REFRESH MATERIALIZED VIEW CONCURRENTLY?

No, I used refresh without concurrently

[riksking]

I tried this approach but cannot reproduce the error. Can you show the full timetable.execution_log including client_name. I suspect you have more than one parallel worker not interfering with each other

With sleepy_func I also can't reproduce it.

Here full timetable.execution_log

"chain_id"	"task_id"	"txid"	"last_run"	"finished"	"pid"	"returncode"	"ignore_error"	"kind"	"command"	"output"	"client_name"
5	6	3381757754	"2024-08-09 16:39:44.573763+03"	"2024-08-09 16:39:58.372841+03"	1329497618	0	true	"SQL"	"REFRESH MATERIALIZED VIEW ""CALC"".""MV_QUEUES_KPI_INFO_V2_new_stat"""	"REFRESH MATERIALIZED VIEW"	"pg_timetable"
5	6	3381753783	"2024-08-09 16:39:29.572478+03"	"2024-08-09 16:39:44.754639+03"	1329497618	0	true	"SQL"	"REFRESH MATERIALIZED VIEW ""CALC"".""MV_QUEUES_KPI_INFO_V2_new_stat"""	"REFRESH MATERIALIZED VIEW"	"pg_timetable"

[pashagolub]

Ough. Mystery. Would you please run the session with --log-level=debug and provide the output. It would be great if you can leave only this chain active for debugging purposes.
Thanks in advance!