Course / Reading data / Lesson 15

Lesson 15 · Reading data

Parameters and macros

Pass values to a query safely and use the macros `#IN_RANGE`, `#PERIOD`, `#BAG` and `#BAGCOLS`.

about 7 minutes · screenplay estimate9 scenesPrerequisites: lesson 14

Development course · Reading edition · API compatibility

Scene 01

Two ways to pass a value

A parameter is written in SQL as a colon followed by a name. Colon minimum is a parameter. You give its value in one of two ways. The first is a keyword argument of query. The second is the sqlparams mapping. Both queries on the slide compile to the same SQL and the same parameters. The value never enters the SQL text. On PostgreSQL the placeholder is written in the driver format, percent, parenthesis, name. Use sqlparams when a parameter name is the same as a query option, or when the names are built at run time. Giving the same name in both places raises ValueError.

Example & screen notes

Slide title "Parameters". Code:

invoice = db.table('invc.invoice')

invoice.query('$inv_number', '$total > :minimum', minimum=100)
invoice.query('$inv_number', '$total > :minimum', sqlparams={'minimum': 100})

Below, the output of query.test() for both:

('SELECT "t0"."inv_number" AS "inv_number"\n FROM "invc"."invc_invoice" AS "t0"\n'
 'WHERE ("t0"."total" > %(minimum)s) AND ("t0"."__del_ts" IS NULL)',
 {'minimum': 100})

Scene 02

A keyword that the query does not use

In this example the keyword is misspelled: minimun instead of minimum. The query still runs. At compilation asqueel emits a warning of class UnusedQueryParameterWarning. The warning names the unused keyword and the parameters that the query uses. The warning points at your line of code. The check applies to keyword arguments only. An unused key in the sqlparams mapping produces no warning. Colon minimum has no value, so it is bound as NULL. The comparison with NULL is never true, and the result is an empty list. The query does not raise an error for a missing parameter.

Example & screen notes

Slide title "Unused and missing parameters". Code:

rows = invoice.query('$inv_number', '$total > :minimum', minimun=100).fetch()
# UnusedQueryParameterWarning: query keyword 'minimun' is not used by the query;
#   its parameters are: minimum
rows    # []

Bullets: - unused keyword: UnusedQueryParameterWarning, the query runs - unused key in sqlparams: no warning - missing parameter: bound as NULL

Scene 03

Collections: IN and ANY

To compare a column with several values, write IN colon name and pass a list, a tuple or a set. Preparation writes one placeholder for each member, numbered from zero. NOT IN works the same way. With an empty collection on PostgreSQL, the IN condition becomes FALSE and the NOT IN condition becomes TRUE. On PostgreSQL you can also write equals ANY, with the parameter in parentheses. Then the list is bound as one array parameter. The two forms do not mix. Do not use the same parameter in IN position and in ANY position in one query; PostgreSQL then reports a missing parameter. A string after IN is not expanded, and PostgreSQL reports a syntax error.

Example & screen notes

Slide title "Collections". Code:

invoice.query('$inv_number', '$inv_number IN :numbers', numbers=['A1', 'B2'])
# WHERE ("t0"."inv_number" IN(%(numbers_0)s,%(numbers_1)s)) ...
# rows: A1, B2

invoice.query('$inv_number', '$inv_number NOT IN :numbers', numbers=['A1', 'B2'])
# rows: A2, B1

invoice.query('$inv_number', '$inv_number = ANY(:numbers)', numbers=['A1', 'B2'])
# WHERE ("t0"."inv_number" = ANY(%(numbers)s)) ...     PostgreSQL only

Bullets: - IN :name / NOT IN :name with a list, tuple or set: one parameter per member - empty collection on PostgreSQL: IN becomes FALSE, NOT IN becomes TRUE - = ANY(:name): one array parameter

Scene 04

Symbolic parameters

The where clause has one more rule. When a parameter value is a string that starts with a dollar sign and names a column of the table, the compiler writes that column into the SQL instead of binding the value. The same happens for a value that starts with an at sign and names an existing relation. The first query on the slide compares gross_total with total. This applies with keywords and with sqlparams alike. A dollar value that names no column stays an ordinary bound value. If your value comes from outside, such as user input, and it can start with a dollar or an at sign, prefix it with a backslash. The backslash is removed before execution, and the text is bound as a plain string.

Example & screen notes

Slide title "A value that names a column". Code:

invoice.query('$inv_number', '$gross_total > :other', other='$total')
# WHERE ("t0"."gross_total" > "t0"."total")

invoice.query('$inv_number', '$inv_number = :other', other=r'\$total')
# WHERE ("t0"."inv_number" = %(other)s)      value bound: '$total'

Bullets: - a WHERE value '$col' or '@rel.col' that exists in the table is compared as a column - a value that names nothing stays a bound value - prefix a backslash to bind the text literally

Scene 05

Environment parameters

A parameter whose name starts with env underscore reads its value from the environment. Colon env_customer_id reads the environment key customer_id. The value is read at every execution, not at compilation. If you pass env_customer_id as a parameter, your value is used and the environment is not read. If neither exists, the parameter is NULL. Lesson seventeen covers the environment.

Example & screen notes

Slide title ":env_ parameters". Code:

query = invoice.query('$inv_number', '$customer_id = :env_customer_id')

with db.temp_env(customer_id=ada_id):
    query.fetch()          # A1, A2

invoice.query('$inv_number', '$customer_id = :env_customer_id',
              env_customer_id=bob_id).fetch()     # B1, B2

Scene 06

#IN_RANGE

A macro starts with a hash sign. The compiler replaces it with SQL before execution. hash IN_RANGE takes a value and two bounds. Both bounds are inclusive. A NULL bound leaves that side open. In the first query the high bound is None, so the condition means total greater than or equal to one hundred. With two NULL bounds the condition is always true. hash IN_RANGE is expanded in the where clause and in the sql_formula of a formulaColumn. In order_by, group_by, having or the columns, it raises UnsupportedFeatureError.

Example & screen notes

Slide title "#IN_RANGE(value, low, high)". Code:

invoice.query('$inv_number', '#IN_RANGE($total, :low, :high)', low=100, high=None)
# rows: A2, B1
invoice.query('$inv_number', '#IN_RANGE($total, :low, :high)', low=60, high=600)
# rows: A2, B2

Bullets: - bounds are inclusive - a NULL bound leaves that side open; two NULL bounds: always true - allowed in WHERE and in a formulaColumn sql_formula

Scene 07

#PERIOD

hash PERIOD takes a date field and a parameter that holds a period in words, such as this month or last month. The colon before the parameter name is optional. The compiler parses the text and binds two dates, period_from and period_to. Parsing uses the workdate and the locale of the environment. The words must be in the language of the locale; with locale en, this month is valid and an Italian phrase raises an error. Set the locale explicitly. The default locale comes from the process, and a value that has no language file raises an error. A single day gives an equality. A closed period gives BETWEEN. An empty text gives true. The dates are computed once, when the query compiles. hash PERIOD is expanded only in the where clause.

Example & screen notes

Slide title "#PERIOD(field, :period)". Code:

from datetime import date

with db.temp_env(workdate=date(2026, 10, 20), locale='en'):
    invoice.query('$inv_number', '#PERIOD($date, :period)', period='this month')
    # WHERE ( ("t0"."date" BETWEEN %(period_from)s AND %(period_to)s) )
    # period_from = 2026-10-01, period_to = 2026-10-31   rows: A2, B1, B2
    invoice.query('$inv_number', '#PERIOD($date, :period)', period='today')
    # WHERE ( "t0"."date" = %(period_from)s )

Bullets: - the period text is parsed at compilation, with workdate and locale of the environment - one day: =; closed interval: BETWEEN; open interval: >= or <=; empty text: true - allowed in WHERE only

Scene 08

#BAG and #BAGCOLS

The last two macros work on X columns, in the query columns only. In the SQL, hash BAG leaves the plain column. After the fetch, asqueel turns the XML text into a Bag object of the genro_bag library. A NULL value gives an empty Bag. hash BAGCOLS spreads the document instead. Each leaf becomes a column named after the field and the path of the leaf, with dots replaced by underscores. The field itself becomes None. Both macros take a column of the queried table; a relation path raises ValueError. In a where, order_by or formula they raise ValueError. A macro that asqueel does not know, such as hash PREF, raises UnsupportedFeatureError.

Example & screen notes

Slide title "XML columns in the result". Code:

rows = invoice.query('$inv_number, #BAG($details) AS doc', order_by='$inv_number').fetch()
rows[0]['doc']['delivery.city']        # 'Sydney'   (a genro_bag Bag)
rows[1]['doc']                         # empty Bag  (NULL in the database)

rows = invoice.query('$inv_number, #BAGCOLS($details)', where="$inv_number = 'A1'").fetch()
dict(rows[0])
# {'inv_number': 'A1', 'details': None,
#  'details_delivery_city': 'Sydney', 'details_delivery_days': '3'}

Scene 09

Recap

To recap. Parameters come from keywords or from sqlparams, and a misspelled keyword produces a warning. IN expands a list, ANY binds an array, and a dollar or at value compares columns. IN_RANGE and PERIOD work in the where clause; BAG and BAGCOLS work in the columns. The next lesson covers the fetch terminals.

Example & screen notes

Slide title "Recap". Bullets: - :name from keywords or sqlparams; a misspelled keyword gives UnusedQueryParameterWarning - IN :list expands, = ANY(:list) binds an array; $/@ values compare columns - #IN_RANGE and #PERIOD in WHERE; #BAG and #BAGCOLS in the columns

Footer: "Next: lesson 16 — Fetch terminals".