I have recently created a Django application that should automatically scrap data from Twitter and populate a database while displaying some of this content to the site viewers.
One challenge I faced during this process was finding the correct packages and writing the best code to carry out these tasks. Thanks to the large open-source contributors out there and Stack Overflow, I finally managed to do this job.
I will lay out the steps for those who might need this information in the future as my way of giving back to the dev community.
Cron jobs are used for scheduling tasks in a computer program. Since I was using Django, I used Django-Cron.
Django-cron lets you run Django/Python code regularly proving basic plumbing to track and execute tasks
The documentation is however pretty straightforward. I did not face any challenges in following along with it. However, there is one thing I'd like to mention.
If you are using Django 4.0 and above, the package installation from their GitHub repo will not work with the pip install technic. The best way would be to clone the repo and use the code in your project files without placing it in the requirements.txt file.
Clone the repo
Then simple replace the line in
from django.utils.translation import ugettext_lazy as _
from django.utils.translation import gettext_lazy as _
ugettext_lazy is now deprecated and has been replaced by
Note: Make sure you do not have
django-cron in your
requirements.txt file otherwise Heroku will install that package and cause your application to crash during install
Go through the process of deploying Django to Heroku.
On your Heroku Dashboard, navigate to your application and click on the Resources tab.
In the textbox where it reads 'Quickly add add-ons from Elements' just type Heroku Scheduler.
The Heroku Scheduler is a free add-on that will help us do the task. Install the add-on and click on it.
On the scheduler page, Create Job and configure the required inputs.
Select the interval you would like your cronjobs to execute.
Since we used
python manage.py runcrons was our command based on the documentation for executing the cron-jobs.
Under the Run Command input field, paste in the django-cron command for executing cron-jobs.
python manage.py runcrons
Will be our command in this case.
Save the job and that's it.
At this point, all Cron-Jobs will start executing automatically as long as you have configured your Django application properly.
That's an overview of my tasks which I have scheduled by implementing this.
Avoid manual command line execution and begin scheduling today. 😉👍