Website and Phone App Strategy for Tech Companies | Harry Moreno

Google Cloud SQL Provision Cheapest Postgres Database

Fri, 15 Nov 2024 00:00:00 +0000

Here is the command for provisioning the cheapest database on Google Cloud SQL. There were a couple of pitfalls as I’ll explain.

gcloud sql instances create myinstance \
  --project my-project \
  --database-version POSTGRES_16 \
  --tier db-f1-micro \
  --region your-region \
  --edition ENTERPRISE

Notes:

By default you may be instructed in the code labs to run

gcloud sql instances create myinstance --project $PROJECT_ID \
  --database-version POSTGRES_14 --tier db-f1-micro --region $REGION

You can change the postgres engine version to something modern like POSTGRES_16. The versions are listed here. As of this writing postgresql 16.3 is the default version.

When you run

gcloud sql instances create myinstance \
  --project $PROJECT_ID \
  --database-version POSTGRES_16 \
  --tier db-f1-micro \
  --region $REGION

you get the error

ERROR: (gcloud.sql.instances.create) HTTPError 400: Invalid request:
Invalid Tier (db-f1-micro) for (ENTERPRISE_PLUS) Edition.
Use a predefined Tier like db-perf-optimized-N-* instead.
Learn more at https://cloud.google.com/sql/docs/postgres/create-instance#machine-types.

You can bypass this by specifying a lower tier for edition

gcloud sql instances create myinstance \
  --project $PROJECT_ID \
  --database-version POSTGRES_16 \
  --tier db-f1-micro \
  --region $REGION \
  --edition ENTERPRISE

In the pricing calculator you can get the cost of the database down to $10.60 per month. I did this by choosing a shared vcpu database

and lowering the disk size to 16gb.

Delete your database to avoid being billed

gcloud sql instances delete myinstance

Sliding Window in python trapezoid

Sun, 29 Oct 2023 00:00:00 +0000

I often find myself having to reimplement the sliding window algorithm. Here is a basic template for sliding windows. It is a good starting point for adding other features like counting characters or tracking the largest size of the window.

This template takes a basic two-pointer approach. But it reads better from top to bottom and generalizes more than other solutions I have seen. First we increase the window size by shifting the right pointer r. If the window reaches the max size we slide it by incrementing pointer r and the left pointer l. If we reach the right side we shrink the window by increasing the left pointer l.

s = "ABCDEFG"
window_length = 3
l = r = 0
while l < len(s):
    if r - l < window_length and r < len(s):
        r += 1
    elif r - l == window_length and r < len(s):
        r += 1
        l += 1
    else:
        l += 1
    win = s[l:r]
    print(win)

output

A
AB
ABC
BCD
CDE
DEF
EFG
FG
G

We can rewrite this as a generator if you prefer

# sliding_window.py
def sliding_window_generator(s, window_length):
    l = r = 0
    while l < len(s):
        if r - l < window_length and r < len(s):
            r += 1
        elif r - l == window_length and r < len(s):
            r += 1
            l += 1
        else:
            l += 1
        win = s[l:r]
        yield(win)

interaction

$ python -i sliding_window.py
>>> g = sliding_window_generator('ABCDEFG', 3)
>>> next(g)
'A'
>>> print(next(g))
AB
>>> next(g)
'ABC'
>>> next(g)
'BCD'
>>> next(g)
'CDE'
>>> next(g)
'DEF'
>>> next(g)
'EFG'
>>> next(g)
'FG'
>>> next(g)
'G'

Django log slow queries

Tue, 12 Sep 2023 00:00:00 +0000

It is important to keep track of the performance of your application. One popular way to do this is to log slow queries. Below is how to do this in Django python. Your logging config will vary depending on how your django app is setup. On simpler django sites it might be in settings.py

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'filters': {
        'slow_queries': {
            '()': 'django.utils.log.CallbackFilter',
            # output slow queries only, unit are 1s, so 0.3 is 300ms
            # ref 300ms * 0.001 = 0.3, 50ms * 0.001 = 0.05
            'callback': lambda record: record.duration > 0.05
        },
    },
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler',
        },
    },
    'loggers': {
        'django': {
            'handlers': ['console'],
            'level': 'DEBUG',
            'propagate': True,
        },
        'django.db.backends': {
            'handlers': ['console'],
            'level': 'DEBUG',
            'propagate': False,
            'filters': ['slow_queries'],
        },
    },
}

This config has two loggers. One named django and one named django.db.backends. Loggers are the entry point for the logging system in django. We apply our slow query logic only to the django.db.backends handler. After a logger has dealt with a log it applies any filters onto it.

Filters provide extra control over logs. We define one filter in the example above. This filter will allow only slow queries to go through, thus we would log slow queries so that the developer can read them later. The filter applies a python lambda inline that will only let logs with a duration > 50 milliseconds pass through.

This config has one handler console. A handler is a destination for where the django logs should go. This could be a file writer, stdout, another handler, etc.

My review of Pandas 30 day challenge

Wed, 30 Aug 2023 00:00:00 +0000

Recently I completed the Leetcode 30 day Pandas Challenge. Pandas is a popular python library for data analysis and Leetcode has made a set of problems to learn it. Here I share my thoughts on the problems and whether you should try it too.

Some background. Before this challenge I felt I had a good understanding of Pandas. I understood what dataframes and Series are. How pandas builds upon numpy datatypes. I also used pandas professionally in the past to build ETL pipelines.

After the challenge I felt like I had a deeper understanding of pandas. I am much more comfortable renaming columns in a dataframe, changing dataframe datatypes, merging frames sql-style and more comfortable solving tasks I may run into daily.

Based on the SQL problems

The set consists of 30 problems to solve. Behind the scenes the problems are the same as other SQL problems on the website, except you have to solve them with python and pandas. This leads to many sql comments on the problems which can be annoying for someone trying to learn only pandas.

(SQL comments mixed with Pandas comments)

Also because the problem statements are for sql you end up getting very comfortable with the pandas apis. For example in many problems you have to construct the final dataframe with specific column names

def food_delivery(delivery: pd.DataFrame) -> pd.DataFrame:
    df = delivery
    num = len(df)
    immediate = len(df[df['order_date'] == df['customer_pref_delivery_date']])
    return pd.DataFrame({'immediate_percentage': [round(immediate/num*100, 2)]})

If we were a python data analyst we wouldn’t be naming columns like this very often.

Example

One problem is called “Customers who never order” in the Data Filtering section. You first get a schema describing the data

Then you get a task, here it asks to find all customers who never order anything

Then you get some example results when they run your solution on different data

Then you are to write your solution in the editor

The editor is in pandas mode. It is the same throughout the website for java questions, javascript question, ruby, etc. You write your solution and can try it on the tests by clicking Run. When you are happy with the solution you can submit and Leetcode will run your solution on a larger test suite serverside.

My Solution

My solution to this problem is

def find_customers(customers: pd.DataFrame, orders: pd.DataFrame) -> pd.DataFrame:
    pd = customers.merge(orders, left_on='id', right_on='customerId', how='left')
    no_order_ids = pd[pd.isnull().any(axis=1)]['id_x']
    names = customers.query(f'id in {list(no_order_ids)}')
    return names[['name']].rename(columns={'name': 'Customers'})

First I merge the customer and orders table on the ids

pd = customers.merge(orders, left_on='id', right_on='customerId', how='left')

Then I get all ids for the customers that made no orders with .isnull()

no_order_ids = pd[pd.isnull().any(axis=1)]['id_x']

Then we use .query() to get all the customers in the original dataframe that we know have no orders (are in the no_order_ids series)

names = customers.query(f'id in {list(no_order_ids)}')

Finally we narrow down the original names dataframe to get the customer’s name and rename the column to fit the problem requirements.

return names[['name']].rename(columns={'name': 'Customers'})

By the way, there is more than one way to solve each problem. You are allowed to use the full expressivity of python and the pandas apis.

Topics

The 30 pandas questions are split into 6 section.

Data Filtering
String Methods
Data Manipulation
Statistics
Data Aggregation
Data Integration

In the following sections we list the pandas apis we used to solve the problems.

Data Filtering

String Methods

Series.str Family of methods https://github.com/pandas-dev/pandas/blob/v2.0.3/pandas/core/strings/accessor.py#L2962
Adding columns with numpy.where
Regex with Series.str.contains() and Series.str.match()

Data Manipulation

DataFrames’s head() and tail()
Numbering rows with rank() https://devdocs.io/pandas~1/reference/api/pandas.dataframe.rank
Reshaping and pivot tables with melt()
deduplicating with drop_duplicates()

Statistics

constructing dataframes with pd.Dataframe()
Quantizing with pd.cut()

Data Aggregation

Aggregations with groupby()
Named Aggregations with lambdas
Adding new columns with assign()

Data Integration

Promoting a group’s index to a column with reset_index()
sql-style-join with merge()
testing membership with Series.isin()
Negating .isin() with the unary not ~

Conclusion

Overall I would recommend doing this challenge. The questions seem realistic to the tasks a data analyst may encounter in everyday work. The questions force you to use a large amount of the pandas api. The problems are rated as “beginner” pandas difficulty. You could complement this challenge with some Kaggle competitions so you could apply your pandas knowledge.

How to prepare for the Flask Backend Engineer Interview

Mon, 13 Mar 2023 00:00:00 +0000

Python development is great for modern web development. There are several python based web frameworks that can help speed up building your web app. In this post we’ll go over one popular framework, Flask, and some of the topics you might get asked in an interview setting.

How to prepare

Review Flask documentation: It’s important to have a good understanding of the Flask framework and its features. Flask documentation is a great resource to learn about its capabilities, features and how to use them.

Understand Flask Architecture: Flask follows the Model-View-Controller (MVC) architecture. Make sure you understand the basics of the architecture and how it’s implemented in Flask.

Practice Flask basics: Start practicing Flask basics such as creating routes, templates, views, and handling requests. Review the differences between Flask and Django and how they handle these concepts. All of these topics are covered in the quickstart.

Review Flask extensions: Flask has many extensions that can help you build your application more easily. Some examples of popular extensions include Flask-WTF, Flask-Login, Flask-Mail, Flask-SQLAlchemy, and Flask-RESTful. Review the extensions that you might be using and how to use them.

Prepare for database and ORM questions: Flask supports multiple ORMs, such as SQLAlchemy, Peewee, and PonyORM. Prepare yourself with basic ORM concepts and review how to use Flask’s most popular orm SQLAlchemy.

Understand Flask’s security features: Flask has built-in security features, such as password hashing, CSRF protection, and session management. It’s important to understand how these features work and how to use them.

Build sample projects: Build some sample projects using Flask to demonstrate your understanding of the framework. You can start with a simple CRUD application and gradually move on to more complex projects like an ETL pipeline. This will help you gain more experience and confidence in working with Flask.

Common questions

What is Flask? How is it different from other web frameworks?

What is the difference between Flask and Django?

Explain the Model-View-Controller (MVC) architecture in Flask.

What are Flask blueprints? How do they help in structuring a large application?

How do you create a route in Flask? Explain the different types of routes in Flask.

What is Flask’s template engine? How do you use it? Jinja

How does Flask handle sessions? What are some security measures Flask provides?

What is an ORM? What are some popular ORMs used in Flask?

What is Flask-WTF? How do you use it for form validation?

What are Flask extensions? Give some examples.

What is Flask-SQLAlchemy? How do you use it?

How do you handle authentication and authorization in Flask?

How do you deploy a Flask application? What are some popular hosting platforms for Flask?

What are some best practices for Flask development?

Differences between Flask and Django

Complexity: Django is a full-stack web framework, which means it comes with built-in features and modules for various functionalities such as ORM, Admin Interface, Routing, Forms, Authentication, and Security. Flask, on the other hand, is a micro-framework, which means it has less built-in functionality and allows developers to add only the necessary modules and packages as per the project requirements.

Scalability: Django is well-suited for large-scale applications, and its built-in modules and features can help developers build complex applications quickly. Flask, on the other hand, is lightweight and highly customizable, making it more suitable for small to medium-sized applications that require flexibility.

Database: Django has its Object-Relational Mapping (ORM) layer, which allows developers to interact with databases using Python syntax without writing SQL queries explicitly. Flask supports various ORMs like SQLAlchemy, Peewee, PonyORM, and more.

Routing: Django has a built-in routing system that makes it easy to define URL patterns and route requests to specific views. Flask also has a routing system but allows developers to create their routing logic using decorators.

Template Engine: Django has its built-in template engine, whereas Flask allows developers to choose their preferred template engine like Jinja2, Mako, and others.

Community: Django has a large and active community, and there are many third-party libraries and plugins available to use. Flask has a smaller community compared to Django, but it’s growing steadily, and there are many useful extensions available.

In summary, Flask is a micro-framework that provides more flexibility, while Django is a full-stack framework that provides more built-in features and is more suitable for large-scale applications. You may choose Django as the main api for a platform or a Saas. Flask is a better choice to power a specific service within a microservices architecture. The choice of framework depends on the project requirements, team expertise, and development goals.

Monitoring Django with Tracing and OpenTelemetry

Fri, 23 Sep 2022 00:00:00 +0000

Distributed tracing allows to observe requests as they propagate through distributed systems, especially those built using a microservices architecture. In a distributed environment, tracing also helps you understand relationships and interactions between microservices. Distributed tracing gives an insight into how a particular microservice is performing and how that service affects other microservices.

Using tracing, you can break down requests into spans. Span is an operation (unit of work) your app performs handling a request, for example, a database query or a network call.

Trace is a tree of spans that shows the path that a request makes through an app. Root span is the first span in a trace.

To learn more about tracing, see Distributed tracing using OpenTelemetry.

What is OpenTelemetry?

OpenTelemetry is a vendor-neutral standard that allows you to collect and export traces, logs, and metrics.

OpenTelemetry is available for most programming languages and allows to send performance data to any tracing tool of your choice.

OpenTelemetry is a community-driven open source project that offers several components:

OpenTelemetry API is a programming interface that you can use to instrument code and collect telemetry data.
OpenTelemetry SDK is the official implementation of OpenTelemetry API that processes and exports collected telemetry to backends.
OpenTelemetry Collector is a proxy between your application and a backend. It receives telemetry data, transforms it, and then exports data to backends that can store it permanently. Collector can also act as an agent that pulls telemetry data from monitored systems, for example, Redis or filesystem metrics.
OTLP is the OpenTelemetry protocol used by SDK and Collector to export data to backends or other collectors. As a transport, OTLP can use gRPC (OTLP/gRPC) or HTTP (OTLP/HTTP).

How to use OpenTelemetry?

OpenTelemetry is available for most programming languages and provides interoperability across different languages and environments.

You can get started with OpenTelemetry by following these 5 steps:

Step 1. Install a distributed tracing tool.
Step 2. Browse OpenTelemetry instrumentations to find examples for your framework and libraries.
Step 3. Configure OpenTelemetry SDK to export telemetry data to the tracing tool.
Step 4. Learn about OpenTelemetry Tracing API for your programming language to create custom instrumentations.
Step 5. Install OpenTelemetry Collector to gather infrastructure metrics.

OpenTelemetry Python API

You can create spans using OpenTelemetry Python API like this:

from opentelemetry import trace

tracer = trace.get_tracer("app_or_package_name", "1.0.0")

def insert_user(**kwargs):
    with tracer.start_as_current_span("insert-user") as span:
        return User.objects.create_user(**kwargs)

You can also record attributes and errors:

def insert_user(**kwargs):
    with tracer.start_as_current_span("insert-user") as span:
        if span.is_recording():
            span.set_attribute("enduser.id", kwargs["id"])
            span.set_attribute("enduser.email", kwargs["email"])
        return User.objects.create_user(**kwargs)

OpenTelemetry also has plans to add support for Python Metrics and Python Logs.

Django OpenTelemetry instrumentation

To instrument Django, you need a correspoding OpenTelemetry Django instrumentation:

pip install opentelemetry-instrumentation-django

Django instrumentation uses DJANGO_SETTINGS_MODULE env variable to find settings file. Django defines that variable in manage.py file so you should instrument Django app from that file:

# manage.py

from opentelemetry.instrumentation.django import DjangoInstrumentor

def main():
    # DjangoInstrumentor uses DJANGO_SETTINGS_MODULE to instrument the project.
    # Make sure the var is available before you call the DjangoInstrumentor.
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mysite.settings")

    DjangoInstrumentor().instrument()

For more details, see OpenTelemetry Django guide.

What is Uptrace?

Uptrace is an open source DataDog competitor powered by OpenTelemetry and ClickHouse. It helps developers pinpoint failures and find performance bottlenecks.

Uptrace uses OpenTelemetry to collect observability data and ClickHouse database to store that data. By using an optimized ClickHouse Go scheme, Uptrace allows to significantly reduce storage requirements, improve query performance, and lower monthly SaaS bills.

Uptrace itself is a single Go binary that can process billions of spans and that can be easily installed with a single command using DEB/RPM packages. Uptrace binary and ClickHouse database is all you need to get started.

What’s next?

Next, you can continue exploring OpenTelemetry or start instrumenting your app using popular instrumentations:

How to Test TLS in SWAKS

Sat, 03 Apr 2021 00:00:00 +0000

In this post we detail how to use Swaks for testing emails over tls. This post picks up where the manual page leaves off and more explicitly goes over how to send a test email over tls. Swaks is a popular tool in penetration testing circles, it is written in Perl and according to it’s github history it was first published on December 12th 2001.

Installing

Swaks is available from most popular package managers for installation.

With ubuntu or debian:

$ sudo apt-get install swaks

In a Dockerfile you can append it to the list of dependencies:

RUN apt-get update \
  # for testing smtp
  && apt-get install -y swaks \
  ...

Note Swaks is not available in apk for alpine linux base images. I mention this because alpine is a popular base image for docker containers.

Send email with tls 1.2

This is the heart of the post. To send an email using swaks and tls 1.2 do:

$ swaks \
  --to harry@test.com \
  --server smtp.server.com \
  --from email@address.com \
  --auth-user username \
  --auth-password password \
  --port 587 \
  -tls \
  --tls-protocol tlsv1_2

in my case we had to specify port 587 and a specific from address due to the security preferences on the smtp server. Your mileage may vary.

More use

For completeness here is what I first tried. Running swaks by only specifying the credentials swaks negotiated NTLM (windows NT Lan Manager (a weak protocol)).

$ swaks \
  --to harry@test.com \
  --server smtp.server.com \
  --from email@address.com \
  --auth-user username \
  --auth-password password \
  --port 587

=== Trying smtp.server.com:587...
=== Connected to smtp.server.com.
<- 220 foo.bar.com Microsoft ESMTP MAIL Service ready at Sat, 3 Apr 2021 19:21:16 -0400
-> EHLO foo.bar.com
<- 250-foo.bar.com Hello
<- 250-SIZE 
<- 250-PIPELINING
<- 250-DSN
<- 250-ENHANCEDSTATUSCODES
<- 250-STARTTLS
<- 250-AUTH GSSAPI NTLM
<- 250-8BITMIME
<- 250-BINARYMIME
<- 250 CHUNKING
-> AUTH NTLM
<- 334 NTLM supported
-> foobar
<- foobar
-> foobar
<- 235  Authentication successful
-> MAIL FROM:<test@server.com>
<- 250 2.1.0 Sender OK
-> RCPT TO:<harry@test.com>
<- 250 2.1.5 Recipient OK
-> DATA
<- 354 Start 
-> Date: Sat, 03 Apr 2021 19:21:16 -0400
-> To: harry@test.com
-> From: email@address.com
-> Subject: test Sat, 03 Apr 2021 19:21:16 -0400
-> Message-Id: <foobar@foo.bar.com>
-> X-Mailer: swaks v20170101.0 jetmore.org/john/code/swaks/
->
-> This is a test mailing
->
-> .
<- 250 2.6.0 <foobar@foo.bar.com> Queued mail for delivery
-> QUIT
<- 221 2.0.0 Service closing transmission channel
=== Connection closed with remote host.

OpenSSL

If for whatever reason you cannot install swaks (for example on alpine linux) you can try the openssl command.

$ openssl s_client -no_tls1 -no_tls1_1 -no_tls1_2 -connect your.mail.host:port

Cybersecurity Tips that Could Save Your Small Business

Thu, 11 Mar 2021 00:00:00 +0000

Cybercriminals like to go after small businesses because they tend to be vulnerable and unprotected compared to larger companies. In fact, nearly half of all cyber attacks target small businesses! Did you know that most small companies that experience a cyber attack go out of business within just six months? Fortunately, there are several precautions you can take to protect your customers’ data and safeguard the future of your company.

Hire Professional Help

If your business does experience a cyber attack and you lose valuable data, don’t try to fix the problem yourself. Many small businesses don’t have in-house IT professionals, so business owners often attempt data recovery on their own. However, recovering lost data is an extremely technical process. If you lose valuable business data to an email scam or phishing attack, it’s best to hire an expert to remedy the situation. Consider hiring the help of a data recovery company to get your business back up and running as quickly as possible and avoid major productivity losses. For example, eSentire, iThemes Security Pro, and FireEye are among the trusted names in security solutions that can help protect your small business.

Educate Yourself and Your Employees

According to CNBC, employee negligence is the greatest cybersecurity risk to American businesses. Employees can put businesses at risk in a variety of ways, from leaving their computers unlocked and unattended to working remotely on unsecured networks. Take some time to educate your employees on good cybersecurity habits. Encourage your employees to create long, complex passwords and change them regularly. Your employees should also know how to recognize phishing emails and social engineering attacks. You may even have to implement policies to ensure that employees’ personal devices are kept secure.

If you (or one of your employees) really want to become an expert in cybersecurity, you could earn a master’s in information technology so you can learn how to better thwart viruses, malware, and scams that could harm your business. Earning a degree online allows you to learn on your own time so you can stay focused on running your business.

Secure Your Internet Connection and Wifi Networks

No network is immune to attacks. However, there are ways to make your internet connection and wifi networks more secure. A firewall is important for keeping malicious software out of your business network. If your network already has a firewall, remember that it may need to be enabled and should be updated regularly.

When it comes to wifi, TechRadar recommends using strong encryption with a WPA-protected network. This is much more secure than old wifi connection types. Plus, each person on your network can use their own username and password to get connected. Make sure to set up a separate wifi network for guests. It’s also smart to hide your network name to avoid attracting the attention of cybercriminals.

Install Antivirus Software

Network security isn’t enough to prevent external threats from infiltrating your business. Strong antivirus protection is essential as well. Antivirus software can help keep your business safe from ransomware, spyware, malware, and viruses. This software provides real-time protection, dealing with infected files or malicious programs immediately. Antivirus software also provides regular device scans to search for threats or security weaknesses. Make sure all of your employee devices have up-to-date antivirus software at all times.

Perform Regular Backups

Performing regular data backups is the best way to secure your business against data loss. If an accident or cyber attack results in lost data, you can just restore it from your backups. There are several ways to back up data, but the easiest method is to use a cloud backup service. With a cloud backup service, you don’t have to manage any equipment or worry about keeping your backup data safe, and it’s easy on your budget. Cloud services allow you to automate regular backups to secure offsite servers so your data can be recovered quickly and painlessly in the event of data loss. This will get your business up and running fast, minimizing costly downtime.

Too many small businesses overlook important cybersecurity measures. Many people don’t realize that a single attack or data loss event could spell the end of their business. Protect your most valuable business asset—your data—by adopting smart cybersecurity habits and being prepared with a recovery plan.

Django Find Records in an Hour

Thu, 03 Dec 2020 00:00:00 +0000

Here we show how to select all records that occur in an hour. This is useful for cases where you are running an async task at some interval smaller than a day. My first attempts at doing this type of filtering involved providing a start and end datetime. But there is a way to do it more simply by leveraging some database functions.

I needed to schedule microservice calls based on some business logic. We know with async tasks its better to add a field to your model with a time to be acted on rather than delaying tasks (5. Using a Long countdown or an eta in the Far Future.). So with an hourly task, its pretty natural to want to find all records that should be acted upon at that hour. The following example shows running some operation when an Event starts 🤗.

# models.py

class Event(models.Model):
    start = models.DateTimeField(blank=True, null=True)

# tasks.py

# In your celery setup area
def setup_periodic_tasks(sender, **kwargs):
    sender.add_periodic_task(
        crontab(minute=15),
        do_for_events_this_hour.s(),
        name="do_for_events_this_hour"
    )

# the actual celery task
@app.task()
def do_for_events_this_hour():
    from django.db.models.functions import Now, TruncHour
    from .models import Event

    events_this_hour = (
        Event.objects.annotate(
            hour=TruncHour("start")
        ).filter(hour=TruncHour(Now()))
    )
    for event in events_this_hour:
        print(f"event this hour: {event}")

This basically truncates the datetime start to the nearest hour. And because we run the task every hour we don’t have to worry about the minutes and seconds.

The crontab(minute=15) bit makes the task run at the 15th minute of every hour. The celery crontab api provides an asterisk for the default values of args like hour, day_of_week, etc. So an invocation like

crontab(minute=15)

is evaluated like

crontab(
    minute=15,
    hour='*',
    day_of_week='*',
    day_of_month='*',
    month_of_year='*'
)

which is analagous to 15 * * * *. See in crontab.

My original approach involved tracking start and end times

start = timezone.now()
end = start + datetime.timedelta(hours=1)
...
Event.objects.filter(
    start__gte=start,
    start__lt=end
)

but you have to worry more about edgecases and this might be slower.

Django Email Management Command

Wed, 14 Oct 2020 00:00:00 +0000

Here we share a small django management command that sends an email. This command is useful because it will use the same email credentials the rest of your django api is using to send emails. So you can use this as a sanity check to see if your settings are correct.

# <your app>/management/commands/send_email.py

from django.conf import settings
from django.core.mail import send_mail
from django.core.management.base import BaseCommand

class Command(BaseCommand):
    """
    Sends an email to the provided addresses.
    """

    help = "Sends an email to the provided addresses."

    def add_arguments(self, parser):
        parser.add_argument("emails", nargs="+")

    def handle(self, *args, **options):
        self.stdout.write(f"send_email from: {settings.EMAIL_FROM_FIELD}")
        for email in options["emails"]:
            try:
                send_mail(
                    "Django test email",
                    "a simple email body",
                    settings.EMAIL_FROM_FIELD,
                    [email],
                )
            except BaseException as e:
                self.stdout.write(f"Problem sending email: {e}")

To use this ssh into your api and run:

$ python manage.py send_email foo@gmail.com bar@gmail.com

If your django api has working email smtp settings you should see a test email at the receiving addresses.

In local development you can use something like mailhog to preview the emails locally. It would look like this