I have a use case where I need to send 16000+ users unique mails. To do this I am using mandrill api( djrill). To make it more efficient and optimized I am trying to use send_mass_mail functionality along with celery.
While doing so I was checking the celery logs and found that for each email, the celery worker is creating a new http connection.
[2015-06-22 14:23:30,745: INFO/Worker-4] Starting new HTTPS connection (1): mandrillapp.com [2015-06-22 14:23:31,521: INFO/Worker-4] Starting new HTTPS connection (1): mandrillapp.com [2015-06-22 14:23:32,162: INFO/Worker-4] Starting new HTTPS connection (1): mandrillapp.com
This will be crippling for our network and will consume a lot of bandwidth.
Here is my code to do the same
@shared_task def test_email_mass(merge_vars): messages =  if DEBUG: for k,v in merge_vars.iteritems(): message =  print v['content'] message.append(title) message.append(v['content']) message.append('[email protected]') message.append([k]) messages.append(tuple(message)) send_mass_mail(tuple(messages), fail_silently=False)
I was expecting that the above code will create only a single http connection.
Edit : I tried the following too.
connection = get_connection() if DEBUG: for k,v in merge_vars.iteritems(): message =  msg = EmailMessage(from_email="[email protected]", subject=title) msg.template_name = "standard" msg.to = [k] msg.from_name = from_name messages.append(msg) connection.send_messages(messages) connection.close()
It still shows that it creates a new http connection everytime.