Out of the box, Wagtail comes with privacy options for
pages
and files
(anything stored in a Collection, e.g. documents and images). The
options that come built in
include:
accessible to anyone (aka Public)
accessible to people in specific groups
accessible to anyone who can log into your site at all
accessible if they have a password you enter on that form
That would seem like a fairly comprehensive list, but at work, we often
restrict things to anyone coming from specific IP addresses. So when
we rolled out our new CMS, we had requests for it to support the “on
campus” option - just like the old ones had.
Adding the “On Campus” option comes in two parts: adding it to the
options offered to the editor and then enforcing that restriction when
serving pages or documents.
Adding a New Restriction Choice
The privacy options are defined in the BaseViewRestriction class in
wagtail.core.models. We will be deciding if a browser is coming from
on or off campus with a middleware, so we will not have to add any columns
to the tables defined by classes inheriting from BaseViewRestriction.
But we do need to add “on campus” to the options offered to the site
editor.
To override the stock RESTRICTION_CHOICES from wagtail.core.models.BaseViewRestriction
with our own, we need to monkeypatch the class. We have a number of small
customizations in our wagtail site, so we collect them all into their own app,
wagtail_patches, which we include in INSTALLED_APPS.
And now the monkeypatching code:
That will add the ON_CAMPUS choice to our form. Since there are no additional
parameters needed for this restriction, you wouldn’t think we would have do
make any additional changes to the form or form validations. But as of Django 3,
we also need to patch the model level validations. We do that like this:
Enforcing Our New Restriction
In our setup, we have split enforcement into two parts, a middleware that
determines if a request is “on campus” or not and then code that uses that
information to show or not show the private page or file. We took this
approach because we already have shared library that does the “on campus”
checking. If you do not need to share the code that checks for on vs off
campus, you may want to put that check directly into the code that enforces
the rule.
On Campus Middleware
Define the following middleware somewhere in your project - customizing it
with your own IP addresses.
Then add this to the MIDDLEWARE list in your Django settings file. Since
this middleware is a silent pass through in both directions (only the side
effect of setting the ON_CAMPUS session variable to True or False matters),
you can put this line anywhere in the list.
Updating the Enforcement Code
The meat of the restriction enforcement is in BaseViewRestriction’s accept_request
method, so we need to add our new on-campus check:
What should happen when accept_request returns False? That depends on which
restriction triggers the failure. For example, if a user fails the LOGIN restriction,
they should be directed to log in - but if they fail the ON_CAMPUS restriction,
they should get an error message. The correct actions for the built-in
restriction types are handled in a before_serve_page hook called
check_view_restrictions
Since we already have monkey patched some other hooks, what we did was
to monkey patch check_view_restrictions:
But looking at the source code for the page serve view, I don’t think
we need to replace the existing check_view_restrictions. I think we
can just add an additional before_serve_page hook that returns our
“sorry you need to be on campus to see this page” message. If I were
doing this from scratch, I would put the following code into one of my
wagtail_hooks.py files (either in the wagtail_patches app or in that
app that contains most of my page models).
At work, we need to build a scheduling system. We want to present the
user with a list of possible dates - and then the possible slots on
that date. I don’t want to have all the possible empty slots in the
database so I thought I would have to build them procedurally using
Python.
The code above loops over the days in the range - and then on
available days, loops over the hours in that day and returns a list of
datetimes. There is a lot of ugly adding of Python timedelta objects
and resetting the time to start iterating on a new day. It works - but
the next step, eliminating slots that are already full, is going to be
even uglier - lots of tedious “does this interval overlap with
existing scheduled events”.
When I started looking into how to check the overlap, I started to
looking into checking overlaps in the database - and found that a)
Postgres has a date range data type (tstzrange), b) Django’s Postgres
extensions has a field that wraps the Postgres tstzrange field
(DateTimeRangeField), and c) the Postgres docs even have
an example
of how to create indexes that prevent you from scheduling more than
one person to occupy a specific room at one time. All that ugly
python, turns into:
The only slightly tricky part of that was restricting allowed days to
MWF. I want my constant to use the day names, not the integers
Postgres uses for days of the week. So I needed to import Python’s
calendar module to convert “Monday” to an integer. Python uses 0
for Monday, but Postgres thinks Monday is 1, so add 1. Then it took me
a little while to figure out how to pass a list into the query in a
way that everything is properly interpolated and quoted; the trick:
tuple(allowed_days).
Now I just need to join to my reservations table to exclude slots
where the schedule is already full.
Django has extensive documentation for it’s ORM but somehow I still
end up surprised by some of the queries it builds. The default logging
configuration doesn’t log queries in the way Rails does (in its
development environment) so when a query appears to give the correct
results, I don’t usually check the SQL. But I recently had a page
start to fail; I finally tracked it down to a specific query but
couldn’t immediately see why I was not getting the row I expected so I
printed the query and Django was not building the query I thought it
had been. The python is:
This produces the following SQL (edited slightly to make it easier to read):
Hmmm that’s not what I want. I don’t want 2 subqueries, one for each
condition. I want one subquery, with two two conditions. If I had
wanted 2 subqueries, I would have written 2 excludes, like this:
But both of those QuerySet definitions produce the same SQL. So how
can I produce the following SQL using the Django ORM:
I tried a couple of things using Q but mostly ended up with syntax
errors. Fortunately I finally found this Stack Overflow thread
with references the bug report for this problem AND the solution. You
can force Django to build the desired subquery by writing the subquery
explicitly:
It’s a little verbose, but it is actually a little clearer in some
respects - it is more like a direct python translation of the desired SQL.
One problem that often comes up when you are using an
object-relational mapper is called the N+1 query problem -
inadvertently doing a query and then doing a separate query for the
related objects for each row. When building sites using Ruby on Rails,
the framework logs all SQL queries (while you are in development
mode). So one tends to fix these inefficient queries as you are
developing - if nothing else, in self-defense so you can actually see
the things you care about in your logs.
Django, on the other hand, does not log anything except the timestamp,
request, response_code, and response size. Its default logging
configuration doesn’t log any request parameters or database
queries. So it’s easy to overlook inefficient queries. So when we
finally put a reasonable amount of test data into our staging server,
we found that several of our API endpoints were agonizingly slow. So,
time for some tuning!
Setup
Lots of people use the django debug toolbar but I really prefer log
files. So I installed and configured Django Query Inspector.
That was helpful for identifying some of the worst offenders but for
the real tuning, I needed this stanza to log all database queries:
Once I had that going, I started looking at some of my nested
serializers. With a couple of well placed “select_related”s on the
queries in my views, I was able to get rid of most of the excess
queries but I was consistently seeing an extra query that I couldn’t
figure out - until I started to write up an issue to post on IRC.
The extra query was coming in because I was using DRF’s browsable API
to do my query tuning. The browsable API includes a web form for
experimenting with the create and update actions in a ModelViewSet and
that form has a select menu for each foreign key relationship that
needs to be created. So when I made a request in the browser, I saw:
But when I made the same request using curl, I only see the one join
query that I was expecting:
I used Kitchenplan to
set up my new mac. There is newer configuration option based on
Ansible by the same author -
Superlumic. I would like
to try it but didn’t have time to experiment with this time around.
The big plus for using Kitchenplan was that our small development team
ended up with Macs that are all configured more or less the same
way. Another plus is it installs bash_it which does a lot more shell
configuring than I have ever bothered to do. The only thing I have
found not to like is that it wants to invoke git’s diff tool instead
of the regular unix diff. To shut that off, I just edited the place
where that was set up. In /etc/bash_it/custom/functions.bash (line
72) I commented out: