[2023-10-24 14:21:20,412: WARNING/ForkPoolWorker-2] /home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host '15c2-220-158-159-149.ngrok-free.app'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( [2023-10-24 14:21:51,119: WARNING/ForkPoolWorker-2] /home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host '15c2-220-158-159-149.ngrok-free.app'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( [2023-10-24 14:22:21,798: WARNING/ForkPoolWorker-2] /home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host '15c2-220-158-159-149.ngrok-free.app'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( [2023-10-24 14:22:52,380: WARNING/ForkPoolWorker-2] /home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host '15c2-220-158-159-149.ngrok-free.app'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( [2023-10-24 14:23:22,942: WARNING/ForkPoolWorker-2] /home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host '15c2-220-158-159-149.ngrok-free.app'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( [2023-10-24 14:23:54,016: WARNING/ForkPoolWorker-2] /home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py:686: RuntimeWarning: Exception raised outside body: OperationalError('(mysql.connector.errors.OperationalError) MySQL Connection not available.'): Traceback (most recent call last): File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py", line 477, in trace_task R = retval = fun(*args, **kwargs) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py", line 760, in __protected_call__ return self.run(*args, **kwargs) File "/home/infominsolutions/scheduler_infomin_solutions/app.py", line 51, in trigger_webhook raise RuntimeError('Maximum retries occured') RuntimeError: Maximum retries occured During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1810, in _execute_context context = constructor( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1398, in _init_compiled self.cursor = self.create_cursor() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1722, in create_cursor return self.create_default_cursor() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1728, in create_default_cursor return self._dbapi_connection.cursor() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 1489, in cursor return self.dbapi_connection.cursor(*args, **kwargs) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/mysql/connector/connection_cext.py", line 700, in cursor raise OperationalError("MySQL Connection not available.") mysql.connector.errors.OperationalError: MySQL Connection not available. The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py", line 494, in trace_task I, R, state, retval = on_error(task_request, exc) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py", line 399, in on_error R = I.handle_error_state( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py", line 178, in handle_error_state return { File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py", line 230, in handle_failure task.backend.mark_as_failure( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/backends/base.py", line 167, in mark_as_failure self.store_result(task_id, exc, state, File "/home/infominsolutions/scheduler_infomin_solutions/backend.py", line 22, in store_result raise e File "/home/infominsolutions/scheduler_infomin_solutions/backend.py", line 16, in store_result api_account = session.query(connection.ApiAccount).filter_by(api_key=api_key).first() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 2743, in first return self.limit(1)._iter().first() # type: ignore File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 2842, in _iter result: Union[ScalarResult[_T], Result[_T]] = self.session.execute( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2262, in execute return self._execute_internal( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2144, in _execute_internal result: Result[Any] = compile_state_cls.orm_execute_statement( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/orm/context.py", line 293, in orm_execute_statement result = conn.execute( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1412, in execute return meth( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 516, in _execute_on_connection return connection._execute_clauseelement( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1635, in _execute_clauseelement ret = self._execute_context( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1816, in _execute_context self._handle_dbapi_exception( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2339, in _handle_dbapi_exception raise sqlalchemy_exception.with_traceback(exc_info[2]) from e File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1810, in _execute_context context = constructor( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1398, in _init_compiled self.cursor = self.create_cursor() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1722, in create_cursor return self.create_default_cursor() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1728, in create_default_cursor return self._dbapi_connection.cursor() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 1489, in cursor return self.dbapi_connection.cursor(*args, **kwargs) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/mysql/connector/connection_cext.py", line 700, in cursor raise OperationalError("MySQL Connection not available.") sqlalchemy.exc.OperationalError: (mysql.connector.errors.OperationalError) MySQL Connection not available. [SQL: SELECT api_account.id AS api_account_id, api_account.api_key AS api_account_api_key FROM api_account WHERE api_account.api_key = %(api_key_1)s LIMIT %(param_1)s] [parameters: [{}]] (Background on this error at: https://sqlalche.me/e/20/e3q8) warn(RuntimeWarning( [2023-10-24 14:23:54,104: ERROR/ForkPoolWorker-2] Task scheduler_infomin_solutions.app.trigger_webhook[9c0e4bbe-e0d9-440d-adc7-1bbcc55ad140] raised unexpected: OperationalError('(mysql.connector.errors.OperationalError) MySQL Connection not available.') Traceback (most recent call last): File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py", line 477, in trace_task R = retval = fun(*args, **kwargs) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py", line 760, in __protected_call__ return self.run(*args, **kwargs) File "/home/infominsolutions/scheduler_infomin_solutions/app.py", line 51, in trigger_webhook raise RuntimeError('Maximum retries occured') RuntimeError: Maximum retries occured During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1810, in _execute_context context = constructor( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1398, in _init_compiled self.cursor = self.create_cursor() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1722, in create_cursor return self.create_default_cursor() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1728, in create_default_cursor return self._dbapi_connection.cursor() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 1489, in cursor return self.dbapi_connection.cursor(*args, **kwargs) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/mysql/connector/connection_cext.py", line 700, in cursor raise OperationalError("MySQL Connection not available.") mysql.connector.errors.OperationalError: MySQL Connection not available. The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py", line 494, in trace_task I, R, state, retval = on_error(task_request, exc) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py", line 399, in on_error R = I.handle_error_state( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py", line 178, in handle_error_state return { File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py", line 230, in handle_failure task.backend.mark_as_failure( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/backends/base.py", line 167, in mark_as_failure self.store_result(task_id, exc, state, File "/home/infominsolutions/scheduler_infomin_solutions/backend.py", line 22, in store_result raise e File "/home/infominsolutions/scheduler_infomin_solutions/backend.py", line 16, in store_result api_account = session.query(connection.ApiAccount).filter_by(api_key=api_key).first() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 2743, in first return self.limit(1)._iter().first() # type: ignore File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 2842, in _iter result: Union[ScalarResult[_T], Result[_T]] = self.session.execute( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2262, in execute return self._execute_internal( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2144, in _execute_internal result: Result[Any] = compile_state_cls.orm_execute_statement( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/orm/context.py", line 293, in orm_execute_statement result = conn.execute( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1412, in execute return meth( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 516, in _execute_on_connection return connection._execute_clauseelement( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1635, in _execute_clauseelement ret = self._execute_context( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1816, in _execute_context self._handle_dbapi_exception( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2339, in _handle_dbapi_exception raise sqlalchemy_exception.with_traceback(exc_info[2]) from e File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1810, in _execute_context context = constructor( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1398, in _init_compiled self.cursor = self.create_cursor() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1722, in create_cursor return self.create_default_cursor() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1728, in create_default_cursor return self._dbapi_connection.cursor() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 1489, in cursor return self.dbapi_connection.cursor(*args, **kwargs) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/mysql/connector/connection_cext.py", line 700, in cursor raise OperationalError("MySQL Connection not available.") sqlalchemy.exc.OperationalError: (mysql.connector.errors.OperationalError) MySQL Connection not available. [SQL: SELECT api_account.id AS api_account_id, api_account.api_key AS api_account_api_key FROM api_account WHERE api_account.api_key = %(api_key_1)s LIMIT %(param_1)s] [parameters: [{}]] (Background on this error at: https://sqlalche.me/e/20/e3q8) [2023-10-24 14:25:12,128: WARNING/ForkPoolWorker-2] /home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host '15c2-220-158-159-149.ngrok-free.app'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( [2023-10-24 14:25:42,756: WARNING/ForkPoolWorker-2] /home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host '15c2-220-158-159-149.ngrok-free.app'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( [2023-10-24 14:26:13,430: WARNING/ForkPoolWorker-2] /home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host '15c2-220-158-159-149.ngrok-free.app'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( [2023-10-24 14:26:44,033: WARNING/ForkPoolWorker-2] /home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host '15c2-220-158-159-149.ngrok-free.app'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( [2023-10-24 14:27:14,652: WARNING/ForkPoolWorker-2] /home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host '15c2-220-158-159-149.ngrok-free.app'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( [2023-10-24 14:27:45,200: ERROR/ForkPoolWorker-2] Task scheduler_infomin_solutions.app.trigger_webhook[f527c6e9-87df-424d-b34d-87ae97a37670] raised unexpected: RuntimeError('Maximum retries occured') Traceback (most recent call last): File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py", line 477, in trace_task R = retval = fun(*args, **kwargs) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py", line 760, in __protected_call__ return self.run(*args, **kwargs) File "/home/infominsolutions/scheduler_infomin_solutions/app.py", line 51, in trigger_webhook raise RuntimeError('Maximum retries occured') RuntimeError: Maximum retries occured [2023-10-24 14:27:45,276: WARNING/ForkPoolWorker-2] /home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host '15c2-220-158-159-149.ngrok-free.app'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( [2023-10-24 14:28:15,867: WARNING/ForkPoolWorker-2] /home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host '15c2-220-158-159-149.ngrok-free.app'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( [2023-10-24 14:28:46,629: WARNING/ForkPoolWorker-2] /home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host '15c2-220-158-159-149.ngrok-free.app'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( [2023-10-24 14:29:17,253: WARNING/ForkPoolWorker-2] /home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host '15c2-220-158-159-149.ngrok-free.app'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( [2023-10-24 14:29:47,879: WARNING/ForkPoolWorker-2] /home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host '15c2-220-158-159-149.ngrok-free.app'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( [2023-10-24 14:30:18,429: ERROR/ForkPoolWorker-2] Task scheduler_infomin_solutions.app.trigger_webhook[cfd454b9-1aee-4da2-b4d7-c497160ded06] raised unexpected: RuntimeError('Maximum retries occured') Traceback (most recent call last): File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py", line 477, in trace_task R = retval = fun(*args, **kwargs) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py", line 760, in __protected_call__ return self.run(*args, **kwargs) File "/home/infominsolutions/scheduler_infomin_solutions/app.py", line 51, in trigger_webhook raise RuntimeError('Maximum retries occured') RuntimeError: Maximum retries occured [2024-06-23 02:46:58,304: WARNING/ForkPoolWorker-2] /home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host 'proxy.infomin.solutions'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( [2024-06-23 02:47:30,358: WARNING/ForkPoolWorker-2] /home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host 'proxy.infomin.solutions'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( [2024-06-23 02:48:02,472: WARNING/ForkPoolWorker-2] /home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host 'proxy.infomin.solutions'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( [2024-06-23 02:48:34,448: WARNING/ForkPoolWorker-2] /home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host 'proxy.infomin.solutions'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( [2024-06-23 02:49:06,568: WARNING/ForkPoolWorker-2] /home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1095: InsecureRequestWarning: Unverified HTTPS request is being made to host 'proxy.infomin.solutions'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( [2024-06-23 02:49:38,959: WARNING/ForkPoolWorker-2] /home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py:686: RuntimeWarning: Exception raised outside body: OperationalError('(mysql.connector.errors.OperationalError) MySQL Connection not available.'): Traceback (most recent call last): File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py", line 477, in trace_task R = retval = fun(*args, **kwargs) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py", line 760, in __protected_call__ return self.run(*args, **kwargs) File "/home/infominsolutions/scheduler_infomin_solutions/app.py", line 51, in trigger_webhook except: RuntimeError: Maximum retries occured During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1810, in _execute_context context = constructor( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1398, in _init_compiled self.cursor = self.create_cursor() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1722, in create_cursor return self.create_default_cursor() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1728, in create_default_cursor return self._dbapi_connection.cursor() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 1489, in cursor return self.dbapi_connection.cursor(*args, **kwargs) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/mysql/connector/connection_cext.py", line 700, in cursor raise OperationalError("MySQL Connection not available.") mysql.connector.errors.OperationalError: MySQL Connection not available. The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py", line 494, in trace_task I, R, state, retval = on_error(task_request, exc) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py", line 399, in on_error R = I.handle_error_state( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py", line 178, in handle_error_state return { File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py", line 230, in handle_failure task.backend.mark_as_failure( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/backends/base.py", line 167, in mark_as_failure self.store_result(task_id, exc, state, File "/home/infominsolutions/scheduler_infomin_solutions/backend.py", line 22, in store_result raise e File "/home/infominsolutions/scheduler_infomin_solutions/backend.py", line 16, in store_result api_account = session.query(connection.ApiAccount).filter_by(api_key=api_key).first() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 2743, in first return self.limit(1)._iter().first() # type: ignore File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 2842, in _iter result: Union[ScalarResult[_T], Result[_T]] = self.session.execute( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2262, in execute return self._execute_internal( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2144, in _execute_internal result: Result[Any] = compile_state_cls.orm_execute_statement( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/orm/context.py", line 293, in orm_execute_statement result = conn.execute( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1412, in execute return meth( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 516, in _execute_on_connection return connection._execute_clauseelement( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1635, in _execute_clauseelement ret = self._execute_context( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1816, in _execute_context self._handle_dbapi_exception( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2339, in _handle_dbapi_exception raise sqlalchemy_exception.with_traceback(exc_info[2]) from e File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1810, in _execute_context context = constructor( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1398, in _init_compiled self.cursor = self.create_cursor() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1722, in create_cursor return self.create_default_cursor() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1728, in create_default_cursor return self._dbapi_connection.cursor() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 1489, in cursor return self.dbapi_connection.cursor(*args, **kwargs) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/mysql/connector/connection_cext.py", line 700, in cursor raise OperationalError("MySQL Connection not available.") sqlalchemy.exc.OperationalError: (mysql.connector.errors.OperationalError) MySQL Connection not available. [SQL: SELECT api_account.id AS api_account_id, api_account.api_key AS api_account_api_key FROM api_account WHERE api_account.api_key = %(api_key_1)s LIMIT %(param_1)s] [parameters: [{}]] (Background on this error at: https://sqlalche.me/e/20/e3q8) warn(RuntimeWarning( [2024-06-23 02:49:39,116: ERROR/ForkPoolWorker-2] Task scheduler_infomin_solutions.app.trigger_webhook[34e116d7-136f-485b-be51-a81b0fff97b9] raised unexpected: OperationalError('(mysql.connector.errors.OperationalError) MySQL Connection not available.') Traceback (most recent call last): File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py", line 477, in trace_task R = retval = fun(*args, **kwargs) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py", line 760, in __protected_call__ return self.run(*args, **kwargs) File "/home/infominsolutions/scheduler_infomin_solutions/app.py", line 51, in trigger_webhook except: RuntimeError: Maximum retries occured During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1810, in _execute_context context = constructor( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1398, in _init_compiled self.cursor = self.create_cursor() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1722, in create_cursor return self.create_default_cursor() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1728, in create_default_cursor return self._dbapi_connection.cursor() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 1489, in cursor return self.dbapi_connection.cursor(*args, **kwargs) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/mysql/connector/connection_cext.py", line 700, in cursor raise OperationalError("MySQL Connection not available.") mysql.connector.errors.OperationalError: MySQL Connection not available. The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py", line 494, in trace_task I, R, state, retval = on_error(task_request, exc) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py", line 399, in on_error R = I.handle_error_state( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py", line 178, in handle_error_state return { File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/app/trace.py", line 230, in handle_failure task.backend.mark_as_failure( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/celery/backends/base.py", line 167, in mark_as_failure self.store_result(task_id, exc, state, File "/home/infominsolutions/scheduler_infomin_solutions/backend.py", line 22, in store_result raise e File "/home/infominsolutions/scheduler_infomin_solutions/backend.py", line 16, in store_result api_account = session.query(connection.ApiAccount).filter_by(api_key=api_key).first() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 2743, in first return self.limit(1)._iter().first() # type: ignore File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 2842, in _iter result: Union[ScalarResult[_T], Result[_T]] = self.session.execute( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2262, in execute return self._execute_internal( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2144, in _execute_internal result: Result[Any] = compile_state_cls.orm_execute_statement( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/orm/context.py", line 293, in orm_execute_statement result = conn.execute( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1412, in execute return meth( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 516, in _execute_on_connection return connection._execute_clauseelement( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1635, in _execute_clauseelement ret = self._execute_context( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1816, in _execute_context self._handle_dbapi_exception( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2339, in _handle_dbapi_exception raise sqlalchemy_exception.with_traceback(exc_info[2]) from e File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1810, in _execute_context context = constructor( File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1398, in _init_compiled self.cursor = self.create_cursor() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1722, in create_cursor return self.create_default_cursor() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1728, in create_default_cursor return self._dbapi_connection.cursor() File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 1489, in cursor return self.dbapi_connection.cursor(*args, **kwargs) File "/home/infominsolutions/scheduler_infomin_solutions/venv/lib/python3.8/site-packages/mysql/connector/connection_cext.py", line 700, in cursor raise OperationalError("MySQL Connection not available.") sqlalchemy.exc.OperationalError: (mysql.connector.errors.OperationalError) MySQL Connection not available. [SQL: SELECT api_account.id AS api_account_id, api_account.api_key AS api_account_api_key FROM api_account WHERE api_account.api_key = %(api_key_1)s LIMIT %(param_1)s] [parameters: [{}]] (Background on this error at: https://sqlalche.me/e/20/e3q8)