Django 3.1 SearchQuery now supports ‘websearch’ search type on PostgreSQL 11+

../../../../_images/django_search_simon_willison.png

https://twitter.com/simonw/status/1287155005095469056?s=20

SearchQuery description

SearchQuery translates the terms the user provides into a search query object that the database compares to a search vector .

By default, all the words the user provides are passed through the stemming algorithms, and then it looks for matches for all of the resulting terms.

  • If search_type is ‘plain’ , which is the default, the terms are treated as separate keywords.

  • If search_type is ‘phrase’ , the terms are treated as a single phrase.

  • If search_type is ‘raw’ , then you can provide a formatted search query with terms and operators.

  • If search_type is ‘websearch’ , then you can provide a formatted search query, similar to the one used by web search engines .

‘websearch’ requires PostgreSQL ≥ 11. Read PostgreSQL’s Full Text Search docs to learn about differences and syntax.

Examples

>>> from django.contrib.postgres.search import SearchQuery
>>> SearchQuery('red tomato')  # two keywords
>>> SearchQuery('tomato red')  # same results as above
>>> SearchQuery('red tomato', search_type='phrase')  # a phrase
>>> SearchQuery('tomato red', search_type='phrase')  # a different phrase
>>> SearchQuery("'tomato' & ('red' | 'green')", search_type='raw')  # boolean operators
>>> SearchQuery("'tomato' ('red' OR 'green')", search_type='websearch')  # websearch operators

SearchQuery terms can be combined logically to provide more flexibility:

>>> from django.contrib.postgres.search import SearchQuery
>>> SearchQuery('meat') & SearchQuery('cheese')  # AND
>>> SearchQuery('meat') | SearchQuery('cheese')  # OR
>>> ~SearchQuery('meat')  # NOT

SearchQuery with search_query=”websearch”

../../../../_images/web_search.png

https://ep2020.europython.eu/media/conference/slides/iGXPHck-a-pythonic-full-text-search.pdf

A pythonic full-text search by Paolo Melchiorre ( https://twitter.com/pauloxnet )

Annonce

../../../../_images/euroconf.png

https://twitter.com/pauloxnet/status/1286024244485382144?s=20

Introduction

How to implement full-text search using only Django and PostgreSQL.

Keeping in mind the pythonic principle that “simple is better than complex” we will see how to implement full-text search in a web service using only Django and PostgreSQL and we will analyse the advantages of this solution compared to more complex solutions based on dedicated search engines.

Abstract

A full-text search on a website is the best way to make its contents easily accessible to users because it returns better results and is i n fact used in online search engines or social networks.

The implementation of full-text search can be complex and many adopt the strategy of using dedicated search engines in addition to the database, but in most cases this strategy turns out to be a big problem of architecture and performance.

In this talk we’ll see a pythonic way to implement full-text search on a website using only Django and PostgreSQL, taking advantage of all the innovations introduced in latest years, and we’ll analyse the problems of using additional search engines with examples deriving from my experience (e.g. djangopoject.com or readthedocs.org).

Through this talk you can learn how to add a full-text search on your website, if it’s based on Django and PostgreSQL, or you can learn how to update the search function of your website if you use other search engines

The slides

../../../../_images/web_search.png

https://ep2020.europython.eu/media/conference/slides/iGXPHck-a-pythonic-full-text-search.pdf

../../../../_images/django_search.png

https://ep2020.europython.eu/media/conference/slides/iGXPHck-a-pythonic-full-text-search.pdf

PostgreSQL full-text search in the Django Admin by Simon Willison

Django 3.1 introduces PostgreSQL search_type=”websearch”, which gives you search with advanced operators like “phrase search” -excluding.

James Turk wrote about this here , and it’s also in my weeknotes .

I decided to add it to my Django Admin interface. It was really easy using the get_search_results() model admin method, documented here .

My models already have a search_document full-text search column, as described in Implementing faceted search with Django and PostgreSQL .

So all I needed to add to my ModelAdmin subclasses was this:

 1 def get_search_results(self, request, queryset, search_term):
 2     if not search_term:
 3         return super().get_search_results(
 4             request, queryset, search_term
 5         )
 6     query = SearchQuery(search_term, search_type="websearch")
 7     rank = SearchRank(F("search_document"), query)
 8     queryset = (
 9         queryset
10         .annotate(rank=rank)
11         .filter(search_document=query)
12         .order_by("-rank")
13     )
14     return queryset, False

Here’s the full implementation for my personal blog.