[2023-10-04 18:41:15,968: DEBUG/MainProcess] | Worker: Closing Hub... [2023-10-04 18:41:15,970: DEBUG/MainProcess] | Worker: Closing Pool... [2023-10-04 18:41:15,971: DEBUG/MainProcess] | Worker: Closing Consumer... [2023-10-04 18:41:15,971: DEBUG/MainProcess] | Worker: Stopping Consumer... [2023-10-04 18:41:15,971: DEBUG/MainProcess] | Consumer: Closing Connection... [2023-10-04 18:41:15,971: DEBUG/MainProcess] | Consumer: Closing Events... [2023-10-04 18:41:15,972: DEBUG/MainProcess] | Consumer: Closing Mingle... [2023-10-04 18:41:15,972: DEBUG/MainProcess] | Consumer: Closing Gossip... [2023-10-04 18:41:15,972: DEBUG/MainProcess] | Consumer: Closing Tasks... [2023-10-04 18:41:15,972: DEBUG/MainProcess] | Consumer: Closing Control... [2023-10-04 18:41:15,972: DEBUG/MainProcess] | Consumer: Closing Heart... [2023-10-04 18:41:15,972: DEBUG/MainProcess] | Consumer: Closing event loop... [2023-10-04 18:41:15,972: DEBUG/MainProcess] | Consumer: Stopping event loop... [2023-10-04 18:41:15,972: DEBUG/MainProcess] | Consumer: Stopping Heart... [2023-10-04 18:41:15,974: DEBUG/MainProcess] | Consumer: Stopping Control... [2023-10-04 18:41:15,984: DEBUG/MainProcess] | Consumer: Stopping Tasks... [2023-10-04 18:41:15,984: DEBUG/MainProcess] Canceling task consumer... [2023-10-04 18:41:15,985: DEBUG/MainProcess] | Consumer: Stopping Gossip... [2023-10-04 18:41:15,987: DEBUG/MainProcess] | Consumer: Stopping Mingle... [2023-10-04 18:41:15,987: DEBUG/MainProcess] | Consumer: Stopping Events... [2023-10-04 18:41:15,987: DEBUG/MainProcess] | Consumer: Stopping Connection... [2023-10-04 18:41:15,987: DEBUG/MainProcess] | Worker: Stopping Pool... [2023-10-04 18:41:17,018: DEBUG/MainProcess] | Worker: Stopping Hub... [2023-10-04 18:41:17,019: DEBUG/MainProcess] | Consumer: Shutdown Heart... [2023-10-04 18:41:17,019: DEBUG/MainProcess] | Consumer: Shutdown Control... [2023-10-04 18:41:17,019: DEBUG/MainProcess] | Consumer: Shutdown Tasks... [2023-10-04 18:41:17,019: DEBUG/MainProcess] Canceling task consumer... [2023-10-04 18:41:17,019: DEBUG/MainProcess] Closing consumer channel... [2023-10-04 18:41:17,020: DEBUG/MainProcess] | Consumer: Shutdown Gossip... [2023-10-04 18:41:17,020: DEBUG/MainProcess] | Consumer: Shutdown Events... [2023-10-04 18:41:17,020: DEBUG/MainProcess] | Consumer: Shutdown Connection... [2023-10-04 18:41:17,023: DEBUG/MainProcess] removing tasks from inqueue until task handler finished [2023-10-04 18:41:21,126: DEBUG/MainProcess] | Worker: Preparing bootsteps. [2023-10-04 18:41:21,128: DEBUG/MainProcess] | Worker: Building graph... [2023-10-04 18:41:21,129: DEBUG/MainProcess] | Worker: New boot order: {StateDB, Timer, Hub, Pool, Autoscaler, Beat, Consumer} [2023-10-04 18:41:21,135: DEBUG/MainProcess] | Consumer: Preparing bootsteps. [2023-10-04 18:41:21,136: DEBUG/MainProcess] | Consumer: Building graph... [2023-10-04 18:41:21,153: DEBUG/MainProcess] | Consumer: New boot order: {Connection, Agent, Events, Mingle, Gossip, Tasks, Control, Heart, event loop} [2023-10-04 18:41:21,177: DEBUG/MainProcess] | Worker: Starting Hub [2023-10-04 18:41:21,177: DEBUG/MainProcess] ^-- substep ok [2023-10-04 18:41:21,177: DEBUG/MainProcess] | Worker: Starting Pool [2023-10-04 18:41:21,983: DEBUG/MainProcess] ^-- substep ok [2023-10-04 18:41:21,985: DEBUG/MainProcess] | Worker: Starting Consumer [2023-10-04 18:41:21,986: DEBUG/MainProcess] | Consumer: Starting Connection [2023-10-04 18:41:22,012: INFO/MainProcess] Connected to redis://:**@127.0.0.1:6379/0 [2023-10-04 18:41:22,013: DEBUG/MainProcess] ^-- substep ok [2023-10-04 18:41:22,013: DEBUG/MainProcess] | Consumer: Starting Events [2023-10-04 18:41:22,016: DEBUG/MainProcess] ^-- substep ok [2023-10-04 18:41:22,016: DEBUG/MainProcess] | Consumer: Starting Mingle [2023-10-04 18:41:22,016: INFO/MainProcess] mingle: searching for neighbors [2023-10-04 18:41:23,031: INFO/MainProcess] mingle: all alone [2023-10-04 18:41:23,031: DEBUG/MainProcess] ^-- substep ok [2023-10-04 18:41:23,032: DEBUG/MainProcess] | Consumer: Starting Gossip [2023-10-04 18:41:23,049: DEBUG/MainProcess] ^-- substep ok [2023-10-04 18:41:23,050: DEBUG/MainProcess] | Consumer: Starting Tasks [2023-10-04 18:41:23,083: DEBUG/MainProcess] ^-- substep ok [2023-10-04 18:41:23,084: DEBUG/MainProcess] | Consumer: Starting Control [2023-10-04 18:41:23,101: DEBUG/MainProcess] ^-- substep ok [2023-10-04 18:41:23,102: DEBUG/MainProcess] | Consumer: Starting Heart [2023-10-04 18:41:23,112: DEBUG/MainProcess] ^-- substep ok [2023-10-04 18:41:23,112: DEBUG/MainProcess] | Consumer: Starting event loop [2023-10-04 18:41:23,113: DEBUG/MainProcess] | Worker: Hub.register Pool... [2023-10-04 18:41:23,116: INFO/MainProcess] worker1@vps.infomin.solutions ready. [2023-10-04 18:41:23,117: DEBUG/MainProcess] basic.qos: prefetch_count->32 [2023-10-04 18:55:10,997: DEBUG/MainProcess] pidbox received method hello(from_node='celery@vps.infomin.solutions', revoked={}) [reply_to:{'exchange': 'reply.celery.pidbox', 'routing_key': '5fe684a1-3c7e-386e-953f-984c32f2d48d'} ticket:11a3eb32-c6f4-4485-84d8-dbdb0c0f85a8] [2023-10-04 18:55:10,998: INFO/MainProcess] sync with celery@vps.infomin.solutions [2023-10-04 18:55:12,028: DEBUG/MainProcess] celery@vps.infomin.solutions joined the party [2023-10-04 18:57:10,297: DEBUG/MainProcess] celery@vps.infomin.solutions left [2023-10-04 18:58:39,372: DEBUG/MainProcess] pidbox received method hello(from_node='celery@vps.infomin.solutions', revoked={}) [reply_to:{'exchange': 'reply.celery.pidbox', 'routing_key': '1a16891c-d46e-3da3-8253-5e1671bba923'} ticket:9429479c-13fc-4394-998a-cfea9e4d60a7] [2023-10-04 18:58:39,372: INFO/MainProcess] sync with celery@vps.infomin.solutions [2023-10-04 18:58:40,398: DEBUG/MainProcess] celery@vps.infomin.solutions joined the party [2023-10-04 18:58:43,777: ERROR/MainProcess] Received unregistered task of type 'scheduler_infomin_solutions.app.trigger_webhook'. The message has been ignored and discarded. Did you remember to import the module containing this task? Or maybe you're using relative imports? Please see https://docs.celeryq.dev/en/latest/internals/protocol.html for more information. The full contents of the message body was: b'[["https://d2db-220-158-159-51.ngrok-free.app/test18", "qwerty", {}, "myf654&*^GH768kljugh765*^&b7twe89"], {}, {"callbacks": null, "errbacks": null, "chain": null, "chord": null}]' (179b) The full contents of the message headers: {'lang': 'py', 'task': 'scheduler_infomin_solutions.app.trigger_webhook', 'id': '08b35fee-2d8d-4fb4-a4a1-b89326a6cd78', 'shadow': None, 'eta': '2023-10-04T18:58:43.766454+05:30', 'expires': None, 'group': None, 'group_index': None, 'retries': 0, 'timelimit': [None, None], 'root_id': '08b35fee-2d8d-4fb4-a4a1-b89326a6cd78', 'parent_id': None, 'argsrepr': "('https://d2db-220-158-159-51.ngrok-free.app/test18', 'qwerty', {}, 'myf654&*^GH768kljugh765*^&b7twe89')", 'kwargsrepr': '{}', 'origin': 'gen4189316@vps.infomin.solutions', 'ignore_result': False, 'stamped_headers': None, 'stamps': {}} The delivery info for this task is: {'exchange': '', 'routing_key': 'celery'} Traceback (most recent call last): File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/worker/consumer/consumer.py", line 658, in on_task_received strategy = strategies[type_] KeyError: 'scheduler_infomin_solutions.app.trigger_webhook' [2023-10-04 18:58:43,784: CRITICAL/MainProcess] Unrecoverable error: AttributeError("'Bunch' object has no attribute 'args'") Traceback (most recent call last): File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/worker/consumer/consumer.py", line 658, in on_task_received strategy = strategies[type_] KeyError: 'scheduler_infomin_solutions.app.trigger_webhook' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/worker/worker.py", line 202, in start self.blueprint.start(self) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/bootsteps.py", line 116, in start step.start(parent) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/bootsteps.py", line 365, in start return self.obj.start() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/worker/consumer/consumer.py", line 340, in start blueprint.start(self) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/bootsteps.py", line 116, in start step.start(parent) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/worker/consumer/consumer.py", line 742, in start c.loop(*c.loop_args()) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/worker/loops.py", line 97, in asynloop next(loop) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/asynchronous/hub.py", line 373, in create_loop cb(*cbargs) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/transport/redis.py", line 1341, in on_readable self.cycle.on_readable(fileno) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/transport/redis.py", line 566, in on_readable chan.handlers[type]() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/transport/redis.py", line 971, in _brpop_read self.connection._deliver(loads(bytes_to_str(item)), dest) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/transport/virtual/base.py", line 1017, in _deliver callback(message) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/transport/virtual/base.py", line 639, in _callback return callback(message) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/messaging.py", line 656, in _receive_callback return on_m(message) if on_m else self.receive(decoded, message) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/worker/consumer/consumer.py", line 660, in on_task_received return on_unknown_task(None, message, exc) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/worker/consumer/consumer.py", line 607, in on_unknown_task self.app.backend.mark_as_failure( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/backends/base.py", line 167, in mark_as_failure self.store_result(task_id, exc, state, File "/home/infominsolutions/scheduler_infomin_solutions/backend.py", line 22, in store_result session.rollback() File "/home/infominsolutions/scheduler_infomin_solutions/backend.py", line 15, in store_result api_key = '' AttributeError: 'Bunch' object has no attribute 'args' [2023-10-04 18:58:43,791: DEBUG/MainProcess] | Worker: Closing Hub... [2023-10-04 18:58:43,791: DEBUG/MainProcess] | Worker: Closing Pool... [2023-10-04 18:58:43,791: DEBUG/MainProcess] | Worker: Closing Consumer... [2023-10-04 18:58:43,791: DEBUG/MainProcess] | Worker: Stopping Consumer... [2023-10-04 18:58:43,792: DEBUG/MainProcess] | Consumer: Closing Connection... [2023-10-04 18:58:43,792: DEBUG/MainProcess] | Consumer: Closing Events... [2023-10-04 18:58:43,792: DEBUG/MainProcess] | Consumer: Closing Mingle... [2023-10-04 18:58:43,792: DEBUG/MainProcess] | Consumer: Closing Gossip... [2023-10-04 18:58:43,792: DEBUG/MainProcess] | Consumer: Closing Tasks... [2023-10-04 18:58:43,792: DEBUG/MainProcess] | Consumer: Closing Control... [2023-10-04 18:58:43,792: DEBUG/MainProcess] | Consumer: Closing Heart... [2023-10-04 18:58:43,792: DEBUG/MainProcess] | Consumer: Closing event loop... [2023-10-04 18:58:43,793: DEBUG/MainProcess] | Consumer: Stopping event loop... [2023-10-04 18:58:43,793: DEBUG/MainProcess] | Consumer: Stopping Heart... [2023-10-04 18:58:43,802: DEBUG/MainProcess] | Consumer: Stopping Control... [2023-10-04 18:58:43,805: DEBUG/MainProcess] | Consumer: Stopping Tasks... [2023-10-04 18:58:43,805: DEBUG/MainProcess] Canceling task consumer... [2023-10-04 18:58:43,805: DEBUG/MainProcess] | Consumer: Stopping Gossip... [2023-10-04 18:58:43,807: DEBUG/MainProcess] | Consumer: Stopping Mingle... [2023-10-04 18:58:43,807: DEBUG/MainProcess] | Consumer: Stopping Events... [2023-10-04 18:58:43,807: DEBUG/MainProcess] | Consumer: Stopping Connection... [2023-10-04 18:58:43,808: DEBUG/MainProcess] | Worker: Stopping Pool... [2023-10-04 18:58:44,854: DEBUG/MainProcess] | Worker: Stopping Hub... [2023-10-04 18:58:44,854: DEBUG/MainProcess] | Consumer: Shutdown Heart... [2023-10-04 18:58:44,854: DEBUG/MainProcess] | Consumer: Shutdown Control... [2023-10-04 18:58:44,854: DEBUG/MainProcess] | Consumer: Shutdown Tasks... [2023-10-04 18:58:44,854: DEBUG/MainProcess] Canceling task consumer... [2023-10-04 18:58:44,854: DEBUG/MainProcess] Closing consumer channel... [2023-10-04 18:58:44,854: DEBUG/MainProcess] | Consumer: Shutdown Gossip... [2023-10-04 18:58:44,854: DEBUG/MainProcess] | Consumer: Shutdown Events... [2023-10-04 18:58:44,855: DEBUG/MainProcess] | Consumer: Shutdown Connection... [2023-10-04 18:58:44,862: DEBUG/MainProcess] removing tasks from inqueue until task handler finished [2023-10-04 20:01:12,846: INFO/MainProcess] Connected to redis://:**@127.0.0.1:6379/0 [2023-10-04 20:01:12,855: INFO/MainProcess] mingle: searching for neighbors [2023-10-04 20:01:13,884: INFO/MainProcess] mingle: all alone [2023-10-04 20:01:13,910: INFO/MainProcess] worker1@vps.infomin.solutions ready. [2023-10-04 20:01:13,954: INFO/MainProcess] sync with worker2@vps.infomin.solutions [2023-10-05 01:03:58,822: INFO/MainProcess] Connected to redis://:**@127.0.0.1:6379/0 [2023-10-05 01:03:58,831: INFO/MainProcess] mingle: searching for neighbors [2023-10-05 01:03:59,865: INFO/MainProcess] mingle: all alone [2023-10-05 01:03:59,894: INFO/MainProcess] worker1@vps.infomin.solutions ready. [2023-10-05 01:04:00,016: INFO/MainProcess] sync with worker2@vps.infomin.solutions [2023-10-18 19:10:03,904: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2023-10-21 19:30:42,740: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2023-10-22 19:04:40,611: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2023-10-22 19:06:28,030: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2023-10-22 19:06:36,125: WARNING/MainProcess] Substantial drift from worker2@vps.infomin.solutions may mean clocks are out of sync. Current drift is 45 seconds. [orig: 2023-10-22 19:06:36.057020 recv: 2023-10-22 19:05:51.195693] [2023-10-22 19:11:57,427: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2023-10-22 19:12:28,891: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2023-10-24 14:25:16,986: INFO/MainProcess] Task scheduler_infomin_solutions.app.trigger_webhook[f1a35ebe-d496-4660-83e7-42cb8a31eb81] received [2023-10-24 14:25:21,402: INFO/MainProcess] Task scheduler_infomin_solutions.app.trigger_webhook[4cbe3a5a-7bea-42be-8718-2bb3c843a746] received [2023-10-24 14:25:31,799: INFO/MainProcess] Task scheduler_infomin_solutions.app.trigger_webhook[2bce8150-e268-47a6-848e-326c41229bf3] received [2023-10-24 14:25:39,865: INFO/MainProcess] Task scheduler_infomin_solutions.app.trigger_webhook[1b26d0f0-7379-4c69-93b0-112701afc4e6] received [2023-11-01 13:47:19,851: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2023-11-09 19:20:12,114: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2023-11-16 18:57:54,415: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2023-11-17 19:19:27,531: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2023-11-25 17:09:17,583: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2023-11-27 19:29:52,156: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2023-11-29 18:50:31,763: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2023-12-01 19:30:06,531: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2023-12-08 19:26:04,827: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2023-12-09 18:53:05,757: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2023-12-13 19:21:02,935: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2023-12-16 18:50:34,737: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2023-12-19 19:21:50,707: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2023-12-25 15:21:08,879: INFO/MainProcess] Task scheduler_infomin_solutions.app.trigger_webhook[e4986ca3-d3e1-47f4-8703-16e0cedbfcb2] received [2024-01-02 02:23:38,011: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-01-02 02:24:16,196: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-01-02 02:26:04,712: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-01-02 02:26:11,195: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-01-02 02:26:17,848: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-01-02 02:27:09,344: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-01-18 19:22:55,238: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-01-20 19:11:28,031: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-01-25 17:44:34,782: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-02-03 17:03:00,856: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 2.00 seconds... (1/100) [2024-02-03 17:03:02,874: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 4.00 seconds... (2/100) [2024-02-03 17:03:06,895: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 6.00 seconds... (3/100) [2024-02-03 17:03:12,935: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 8.00 seconds... (4/100) [2024-02-03 17:03:20,946: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 10.00 seconds... (5/100) [2024-02-03 17:03:30,966: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 12.00 seconds... (6/100) [2024-02-03 17:03:42,989: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 14.00 seconds... (7/100) [2024-02-03 17:03:57,035: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 16.00 seconds... (8/100) [2024-02-03 17:04:13,093: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 18.00 seconds... (9/100) [2024-02-03 17:04:31,115: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 20.00 seconds... (10/100) [2024-02-03 17:04:51,137: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 22.00 seconds... (11/100) [2024-02-03 17:05:13,163: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 24.00 seconds... (12/100) [2024-02-03 17:05:37,196: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 26.00 seconds... (13/100) [2024-02-03 17:06:03,234: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 28.00 seconds... (14/100) [2024-02-03 17:06:31,268: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 30.00 seconds... (15/100) [2024-02-03 17:07:01,305: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:07:33,342: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:08:05,380: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:08:37,420: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:09:09,456: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:09:41,494: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:10:13,530: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:10:45,571: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:11:17,607: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:11:49,643: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:12:21,694: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:12:53,733: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:13:25,777: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:13:57,818: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:14:29,853: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:15:01,902: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:15:33,941: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:16:05,978: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:16:38,018: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:17:10,056: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:17:42,098: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:18:14,138: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:18:46,211: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:19:18,248: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:19:50,287: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:20:22,336: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:20:54,373: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:21:26,409: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:21:58,451: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:22:30,488: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:23:02,527: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:23:34,566: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:24:06,604: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:24:38,644: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:25:10,709: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:25:42,747: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:26:14,784: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:26:46,820: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:27:18,856: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:27:50,892: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:28:22,934: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:28:54,974: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:29:27,017: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:29:59,060: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:30:31,100: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:31:03,137: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:31:35,182: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:32:07,220: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:32:39,256: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:33:11,293: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:33:43,332: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:34:15,369: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:34:47,408: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:35:19,445: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:35:51,485: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:36:23,523: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:36:55,564: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:37:27,599: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:37:59,634: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:38:31,671: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:39:03,715: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:39:35,770: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:40:07,815: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:40:39,852: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:41:11,891: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:41:43,930: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:42:15,969: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:42:48,007: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:43:20,043: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:43:52,079: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:44:24,116: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:44:56,154: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:45:28,195: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:46:00,231: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:46:32,266: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:47:04,305: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:47:36,350: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:48:08,388: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:48:40,427: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:49:12,465: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:49:44,508: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:50:16,552: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:50:48,589: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:51:20,625: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:51:52,664: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2024-02-03 17:52:24,702: CRITICAL/MainProcess] Unrecoverable error: OperationalError('Error 111 connecting to 127.0.0.1:6379. Connection refused.') Traceback (most recent call last): File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/transport/virtual/base.py", line 951, in create_channel return self._avail_channels.pop() IndexError: pop from empty list During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/redis/connection.py", line 264, in connect sock = self.retry.call_with_retry( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/redis/retry.py", line 46, in call_with_retry return do() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/redis/connection.py", line 265, in lambda: self._connect(), lambda error: self.disconnect(error) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/redis/connection.py", line 627, in _connect raise err File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/redis/connection.py", line 615, in _connect sock.connect(socket_address) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/connection.py", line 472, in _reraise_as_library_errors yield File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/connection.py", line 459, in _ensure_connection return retry_over_time( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/utils/functional.py", line 318, in retry_over_time return fun(*args, **kwargs) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/connection.py", line 934, in _connection_factory self._connection = self._establish_connection() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/connection.py", line 860, in _establish_connection conn = self.transport.establish_connection() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/transport/virtual/base.py", line 975, in establish_connection self._avail_channels.append(self.create_channel(self)) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/transport/virtual/base.py", line 953, in create_channel channel = self.Channel(connection) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/transport/redis.py", line 741, in __init__ self.client.ping() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/redis/commands/core.py", line 1216, in ping return self.execute_command("PING", **kwargs) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/redis/client.py", line 533, in execute_command conn = self.connection or pool.get_connection(command_name, **options) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/redis/connection.py", line 1086, in get_connection connection.connect() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/redis/connection.py", line 270, in connect raise ConnectionError(self._error_message(e)) redis.exceptions.ConnectionError: Error 111 connecting to 127.0.0.1:6379. Connection refused. The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/worker/worker.py", line 202, in start self.blueprint.start(self) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/bootsteps.py", line 116, in start step.start(parent) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/bootsteps.py", line 365, in start return self.obj.start() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/worker/consumer/consumer.py", line 340, in start blueprint.start(self) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/bootsteps.py", line 116, in start step.start(parent) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/worker/consumer/connection.py", line 21, in start c.connection = c.connect() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/worker/consumer/consumer.py", line 469, in connect conn = self.connection_for_read(heartbeat=self.amqheartbeat) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/worker/consumer/consumer.py", line 475, in connection_for_read return self.ensure_connected( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/worker/consumer/consumer.py", line 526, in ensure_connected conn = conn.ensure_connection( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/connection.py", line 406, in ensure_connection self._ensure_connection(*args, **kwargs) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/connection.py", line 459, in _ensure_connection return retry_over_time( File "/usr/lib64/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/connection.py", line 476, in _reraise_as_library_errors raise ConnectionError(str(exc)) from exc kombu.exceptions.OperationalError: Error 111 connecting to 127.0.0.1:6379. Connection refused. [2024-06-12 21:17:18,693: INFO/MainProcess] Connected to redis://:**@127.0.0.1:6379/0 [2024-06-12 21:17:18,699: INFO/MainProcess] mingle: searching for neighbors [2024-06-12 21:17:19,714: INFO/MainProcess] mingle: all alone [2024-06-12 21:17:19,741: INFO/MainProcess] worker1@vps.infomin.solutions ready. [2024-06-12 21:17:20,402: INFO/MainProcess] sync with worker2@vps.infomin.solutions [2024-06-12 21:17:32,091: INFO/MainProcess] Events of group {task} enabled by remote. [2024-06-12 21:21:33,791: INFO/MainProcess] Task scheduler_infomin_solutions.app.trigger_webhook[98a612c3-a443-44a6-8619-4763bd419fb1] received [2024-06-12 22:22:36,992: INFO/MainProcess] Task scheduler_infomin_solutions.app.trigger_webhook[bbbf7fe4-056d-40cc-8976-19affcfd8d7f] received [2024-06-12 23:43:56,175: INFO/MainProcess] Task scheduler_infomin_solutions.app.trigger_webhook[41532917-f844-48ff-ae73-725ba95b9e2f] received [2024-06-12 23:53:43,639: INFO/MainProcess] Task scheduler_infomin_solutions.app.trigger_webhook[00fe43b7-f99d-4145-bd0c-fa80813a123d] received [2024-06-12 23:59:54,307: INFO/MainProcess] Task scheduler_infomin_solutions.app.trigger_webhook[bbb5646d-5529-4751-b752-8a9ae1a5b01d] received [2024-06-13 00:04:27,044: INFO/MainProcess] Task scheduler_infomin_solutions.app.trigger_webhook[d6009c11-8dca-466e-8c58-bf7b611f403e] received [2024-06-16 03:22:42,626: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-06-16 17:02:57,766: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-06-18 18:11:36,346: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-06-23 02:14:50,434: INFO/MainProcess] Task scheduler_infomin_solutions.app.trigger_webhook[01bc6c79-35b7-4e4d-ae9c-26b311b8c6c4] received [2024-06-23 02:28:46,523: INFO/MainProcess] Task scheduler_infomin_solutions.app.trigger_webhook[0152418a-1907-48c4-9412-f32fe3eee4a2] received [2024-06-23 02:40:31,959: INFO/MainProcess] Task scheduler_infomin_solutions.app.trigger_webhook[d4456dcf-1e16-4886-9299-50afa2587187] received [2024-06-23 02:49:11,515: INFO/MainProcess] Task scheduler_infomin_solutions.app.trigger_webhook[e9a40a74-01e6-4eea-af6a-54b84b286343] received [2024-06-23 02:51:15,292: INFO/MainProcess] Task scheduler_infomin_solutions.app.trigger_webhook[f9fbf8b4-6938-402c-983a-d99e46af9c59] received [2024-06-23 23:43:24,584: INFO/MainProcess] Task scheduler_infomin_solutions.app.trigger_webhook[50b7fedf-63c9-4529-89f0-35d0341867ca] received [2024-06-24 00:08:08,050: INFO/MainProcess] Task scheduler_infomin_solutions.app.trigger_webhook[78f3d5cf-6b5b-44e8-9175-006836378e0f] received [2024-06-24 00:48:33,478: INFO/MainProcess] Task scheduler_infomin_solutions.app.trigger_webhook[d52c1f7e-cedb-4d91-abbc-2c688c95b092] received [2024-06-24 00:53:25,477: INFO/MainProcess] Task scheduler_infomin_solutions.app.trigger_webhook[36a02765-91f7-42c1-9919-072fe67e6e01] received [2024-06-24 01:00:33,068: INFO/MainProcess] Task scheduler_infomin_solutions.app.trigger_webhook[9ab0a6a6-60e3-400c-92ff-c41cd2d007e2] received [2024-06-24 01:01:47,390: INFO/MainProcess] Task scheduler_infomin_solutions.app.trigger_webhook[a76dc6ec-def6-4a02-8c0e-a95c512df853] received [2024-06-24 13:02:09,741: INFO/MainProcess] Task scheduler_infomin_solutions.app.trigger_webhook[58b819c7-c947-4d45-8413-be9f8c053f0f] received [2024-06-24 18:06:48,252: INFO/MainProcess] Connected to redis://:**@127.0.0.1:6379/0 [2024-06-24 18:06:48,263: INFO/MainProcess] mingle: searching for neighbors [2024-06-24 18:06:49,290: INFO/MainProcess] mingle: all alone [2024-06-24 18:06:49,326: INFO/MainProcess] worker1@vps.infomin.solutions ready. [2024-06-24 18:06:49,904: INFO/MainProcess] Events of group {task} enabled by remote. [2024-06-24 18:06:50,228: INFO/MainProcess] sync with worker2@vps.infomin.solutions [2024-07-02 15:25:52,430: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-02 15:26:05,089: WARNING/MainProcess] Substantial drift from worker2@vps.infomin.solutions may mean clocks are out of sync. Current drift is 17 seconds. [orig: 2024-07-02 15:26:05.063161 recv: 2024-07-02 15:25:48.450687] [2024-07-06 03:46:23,777: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-06 03:47:24,083: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-06 03:47:53,032: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-06 03:48:18,868: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-09 23:50:42,756: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-09 23:51:06,034: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-09 23:51:21,252: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-09 23:51:28,144: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-09 23:51:33,620: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-09 23:51:46,685: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-09 23:51:57,476: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-09 23:52:04,492: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-09 23:52:28,326: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-11 06:05:23,351: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-12 01:31:16,371: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-12 07:13:36,386: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-12 07:14:02,908: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-12 07:14:16,668: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-12 07:14:34,475: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-12 07:14:41,836: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-12 07:15:01,043: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-12 07:37:58,533: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-12 07:38:18,381: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-12 15:00:51,761: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-12 15:01:07,692: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-12 15:01:19,571: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-12 15:01:36,571: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-14 07:04:01,335: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-14 07:04:39,276: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-16 07:47:38,254: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-07-16 07:48:40,778: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-08-07 10:45:26,794: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-08-07 10:45:43,945: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-08-07 10:46:12,947: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-08-07 10:46:29,555: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-08-07 10:46:42,305: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-08-16 06:56:37,620: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-08-16 06:57:07,020: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-08-21 23:57:54,926: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-08-22 10:33:16,050: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-08-22 10:33:51,072: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-08-22 10:36:56,531: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-08-23 23:56:24,822: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-08-24 03:28:53,463: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-08-24 04:19:47,043: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-08-24 04:20:09,659: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-08-30 20:12:15,402: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-09-04 17:16:50,201: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2024-09-14 20:52:14,810: INFO/MainProcess] missed heartbeat from worker2@vps.infomin.solutions [2025-03-23 12:15:28,730: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 2.00 seconds... (1/100) [2025-03-23 12:15:30,754: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 4.00 seconds... (2/100) [2025-03-23 12:15:34,762: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 6.00 seconds... (3/100) [2025-03-23 12:15:40,776: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 8.00 seconds... (4/100) [2025-03-23 12:15:48,789: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 10.00 seconds... (5/100) [2025-03-23 12:15:58,807: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 12.00 seconds... (6/100) [2025-03-23 12:16:10,823: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 14.00 seconds... (7/100) [2025-03-23 12:16:24,845: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 16.00 seconds... (8/100) [2025-03-23 12:16:40,864: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 18.00 seconds... (9/100) [2025-03-23 12:16:58,884: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 20.00 seconds... (10/100) [2025-03-23 12:17:18,908: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 22.00 seconds... (11/100) [2025-03-23 12:17:40,933: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 24.00 seconds... (12/100) [2025-03-23 12:18:04,961: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 26.00 seconds... (13/100) [2025-03-23 12:18:30,989: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 28.00 seconds... (14/100) [2025-03-23 12:18:59,024: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 30.00 seconds... (15/100) [2025-03-23 12:19:29,059: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:20:01,094: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:20:33,130: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:21:05,164: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:21:37,200: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:22:09,238: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:22:41,273: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:23:13,310: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:23:45,351: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:24:17,387: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:24:49,429: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:25:21,484: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:25:53,523: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:26:25,560: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:26:57,597: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:27:29,634: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:28:01,671: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:28:33,710: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:29:05,747: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:29:37,784: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:30:09,826: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:30:41,865: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:31:13,904: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:31:45,941: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:32:17,980: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:32:50,019: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:33:22,056: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:33:54,093: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:34:26,129: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:34:58,165: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:35:30,206: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:36:02,241: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:36:34,277: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:37:06,313: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:37:38,349: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:38:10,385: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:38:42,420: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:39:14,455: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:39:46,493: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:40:18,531: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:40:50,567: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:41:22,603: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:41:54,643: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:42:26,682: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:42:58,719: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:43:30,754: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:44:02,789: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:44:34,825: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:45:06,860: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:45:38,896: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:46:10,931: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:46:42,966: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:47:15,003: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:47:47,038: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:48:19,078: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:48:51,114: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:49:23,150: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:49:55,186: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:50:27,224: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:50:59,262: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:51:31,297: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:52:03,335: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:52:35,371: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:53:07,407: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:53:39,445: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:54:11,485: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:54:43,525: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:55:15,570: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:55:47,609: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:56:19,644: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:56:51,680: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:57:23,712: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:57:55,749: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:58:27,786: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:58:59,821: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 12:59:31,859: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 13:00:03,898: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 13:00:35,942: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 13:01:07,980: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 13:01:40,015: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 13:02:12,052: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 13:02:44,087: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 13:03:16,123: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 13:03:48,159: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 13:04:20,195: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-03-23 13:04:52,237: CRITICAL/MainProcess] Unrecoverable error: OperationalError('Error 111 connecting to 127.0.0.1:6379. Connection refused.') Traceback (most recent call last): File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/transport/virtual/base.py", line 951, in create_channel return self._avail_channels.pop() IndexError: pop from empty list During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/redis/connection.py", line 264, in connect sock = self.retry.call_with_retry( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/redis/retry.py", line 46, in call_with_retry return do() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/redis/connection.py", line 265, in lambda: self._connect(), lambda error: self.disconnect(error) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/redis/connection.py", line 627, in _connect raise err File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/redis/connection.py", line 615, in _connect sock.connect(socket_address) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/connection.py", line 472, in _reraise_as_library_errors yield File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/connection.py", line 459, in _ensure_connection return retry_over_time( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/utils/functional.py", line 318, in retry_over_time return fun(*args, **kwargs) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/connection.py", line 934, in _connection_factory self._connection = self._establish_connection() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/connection.py", line 860, in _establish_connection conn = self.transport.establish_connection() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/transport/virtual/base.py", line 975, in establish_connection self._avail_channels.append(self.create_channel(self)) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/transport/virtual/base.py", line 953, in create_channel channel = self.Channel(connection) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/transport/redis.py", line 741, in __init__ self.client.ping() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/redis/commands/core.py", line 1216, in ping return self.execute_command("PING", **kwargs) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/redis/client.py", line 533, in execute_command conn = self.connection or pool.get_connection(command_name, **options) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/redis/connection.py", line 1086, in get_connection connection.connect() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/redis/connection.py", line 270, in connect raise ConnectionError(self._error_message(e)) redis.exceptions.ConnectionError: Error 111 connecting to 127.0.0.1:6379. Connection refused. The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/worker/worker.py", line 202, in start self.blueprint.start(self) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/bootsteps.py", line 116, in start step.start(parent) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/bootsteps.py", line 365, in start return self.obj.start() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/worker/consumer/consumer.py", line 340, in start blueprint.start(self) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/bootsteps.py", line 116, in start step.start(parent) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/worker/consumer/connection.py", line 21, in start c.connection = c.connect() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/worker/consumer/consumer.py", line 469, in connect conn = self.connection_for_read(heartbeat=self.amqheartbeat) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/worker/consumer/consumer.py", line 475, in connection_for_read return self.ensure_connected( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/worker/consumer/consumer.py", line 526, in ensure_connected conn = conn.ensure_connection( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/connection.py", line 406, in ensure_connection self._ensure_connection(*args, **kwargs) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/connection.py", line 459, in _ensure_connection return retry_over_time( File "/usr/lib64/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/connection.py", line 476, in _reraise_as_library_errors raise ConnectionError(str(exc)) from exc kombu.exceptions.OperationalError: Error 111 connecting to 127.0.0.1:6379. Connection refused. [2025-04-10 17:06:38,101: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 2.00 seconds... (1/100) [2025-04-10 17:06:40,113: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 4.00 seconds... (2/100) [2025-04-10 17:06:44,119: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 6.00 seconds... (3/100) [2025-04-10 17:06:50,137: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 8.00 seconds... (4/100) [2025-04-10 17:06:58,147: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 10.00 seconds... (5/100) [2025-04-10 17:07:08,161: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 12.00 seconds... (6/100) [2025-04-10 17:07:20,176: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 14.00 seconds... (7/100) [2025-04-10 17:07:34,208: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 16.00 seconds... (8/100) [2025-04-10 17:07:50,229: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 18.00 seconds... (9/100) [2025-04-10 17:08:08,258: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 20.00 seconds... (10/100) [2025-04-10 17:08:28,285: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 22.00 seconds... (11/100) [2025-04-10 17:08:50,314: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 24.00 seconds... (12/100) [2025-04-10 17:09:14,350: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 26.00 seconds... (13/100) [2025-04-10 17:09:40,379: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 28.00 seconds... (14/100) [2025-04-10 17:10:08,411: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 30.00 seconds... (15/100) [2025-04-10 17:10:38,444: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:11:10,479: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:11:42,516: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:12:14,552: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:12:46,591: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:13:18,626: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:13:50,663: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:14:22,699: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:14:54,737: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:15:26,796: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:15:58,832: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:16:30,868: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:17:02,902: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:17:34,938: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:18:06,974: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:18:39,010: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:19:11,048: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:19:43,083: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:20:15,139: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:20:47,173: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:21:19,209: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:21:51,245: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:22:23,281: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:22:55,320: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:23:27,355: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:23:59,394: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:24:31,430: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:25:03,486: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:25:35,529: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:26:07,565: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:26:39,606: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:27:11,643: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:27:43,678: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:28:15,719: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:28:47,757: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:29:19,792: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:29:51,827: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:30:23,869: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:30:55,905: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:31:27,940: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:31:59,977: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:32:32,016: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:33:04,054: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:33:36,098: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:34:08,137: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:34:40,174: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:35:12,223: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:35:44,266: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:36:16,305: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:36:48,343: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:37:20,379: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:37:52,417: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:38:24,452: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:38:56,490: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:39:28,529: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:40:00,564: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:40:32,605: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:41:04,641: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:41:36,689: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:42:08,727: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:42:40,767: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:43:12,805: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:43:44,842: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:44:16,880: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:44:48,917: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:45:20,957: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:45:52,991: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:46:25,026: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:46:57,066: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:47:29,103: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:48:01,141: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:48:33,176: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:49:05,213: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:49:37,250: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:50:09,294: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:50:41,332: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:51:13,366: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:51:45,402: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:52:17,439: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:52:49,485: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:53:21,523: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:53:53,561: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:54:25,600: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:54:57,635: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:55:29,680: ERROR/MainProcess] consumer: Cannot connect to redis://:**@127.0.0.1:6379/0: Error 111 connecting to 127.0.0.1:6379. Connection refused.. Trying again in 32.00 seconds... (16/100) [2025-04-10 17:56:01,716: CRITICAL/MainProcess] Unrecoverable error: OperationalError('Error 111 connecting to 127.0.0.1:6379. Connection refused.') Traceback (most recent call last): File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/transport/virtual/base.py", line 951, in create_channel return self._avail_channels.pop() IndexError: pop from empty list During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/redis/connection.py", line 264, in connect sock = self.retry.call_with_retry( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/redis/retry.py", line 46, in call_with_retry return do() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/redis/connection.py", line 265, in lambda: self._connect(), lambda error: self.disconnect(error) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/redis/connection.py", line 627, in _connect raise err File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/redis/connection.py", line 615, in _connect sock.connect(socket_address) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/connection.py", line 472, in _reraise_as_library_errors yield File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/connection.py", line 459, in _ensure_connection return retry_over_time( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/utils/functional.py", line 318, in retry_over_time return fun(*args, **kwargs) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/connection.py", line 934, in _connection_factory self._connection = self._establish_connection() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/connection.py", line 860, in _establish_connection conn = self.transport.establish_connection() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/transport/virtual/base.py", line 975, in establish_connection self._avail_channels.append(self.create_channel(self)) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/transport/virtual/base.py", line 953, in create_channel channel = self.Channel(connection) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/transport/redis.py", line 741, in __init__ self.client.ping() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/redis/commands/core.py", line 1216, in ping return self.execute_command("PING", **kwargs) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/redis/client.py", line 533, in execute_command conn = self.connection or pool.get_connection(command_name, **options) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/redis/connection.py", line 1086, in get_connection connection.connect() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/redis/connection.py", line 270, in connect raise ConnectionError(self._error_message(e)) redis.exceptions.ConnectionError: Error 111 connecting to 127.0.0.1:6379. Connection refused. The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/worker/worker.py", line 202, in start self.blueprint.start(self) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/bootsteps.py", line 116, in start step.start(parent) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/bootsteps.py", line 365, in start return self.obj.start() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/worker/consumer/consumer.py", line 340, in start blueprint.start(self) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/bootsteps.py", line 116, in start step.start(parent) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/worker/consumer/connection.py", line 21, in start c.connection = c.connect() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/worker/consumer/consumer.py", line 469, in connect conn = self.connection_for_read(heartbeat=self.amqheartbeat) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/worker/consumer/consumer.py", line 475, in connection_for_read return self.ensure_connected( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/worker/consumer/consumer.py", line 526, in ensure_connected conn = conn.ensure_connection( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/connection.py", line 406, in ensure_connection self._ensure_connection(*args, **kwargs) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/connection.py", line 459, in _ensure_connection return retry_over_time( File "/usr/lib64/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/kombu/connection.py", line 476, in _reraise_as_library_errors raise ConnectionError(str(exc)) from exc kombu.exceptions.OperationalError: Error 111 connecting to 127.0.0.1:6379. Connection refused.