Delay Notification system in Airflow

The Service Level Agreement (SLA) provides the functionality of sending emails in the event a task exceeds its expected time frame from the start of the DAG execution and according to the official documentation and this medium article, Airflow currently doesn’t support this feature in triggered DAGs; in which case, I recommend you open an improvement issue in the Airflow Issue Tracker to request that this feature be supported.

On the other hand, you can use the following approach to validate the SLA after the DAG is finished running via on_success_callback:

def success_function (context):
   s_date = context.get('task_instance').start_date.replace(tzinfo = None)
   e_date = datetime.now().replace(tzinfo = None)
  
   execution_time = e_date - s_date
   sla = timedelta (seconds = 30)
 
   if execution_time> sla:
       print('send_email')
   else:
       print('no send email')

t1 = BashOperator (
   task_id = 'print_date',
   bash_command = 'date',
   dag = dag,
   on_success_callback = success_function
)

Please keep the following considerations in mind if you implement this workaround:

  • The context doesn’t contain the information of the SLA configured in the task; therefore, it is necessary to specify it within the function for each task where you want to validate the SLA
  • Missed SLAs will not appear in the Airflow UI
  • It is necessary to create the logic to send emails

CLICK HERE to find out more related problems solutions.

Leave a Comment

Your email address will not be published.

Scroll to Top