Scene 01
Three policies
A row policy is a table attribute that adds a condition to every SELECT on that table. There are three policies. A partition shows only the rows that belong to the current context. A draft field hides draft rows. A logical-deletion field hides rows that are marked as deleted. Each policy has a query option that changes its effect for one query. Policies are filters on reads. They are not database permissions.
Example & screen notes
Slide title "Row policies". Table:
| Table attribute | Rows hidden by default | Query option |
|---|---|---|
partition_<field>='<path>' |
rows outside current_<path> / allowed_<path> |
ignore_partition |
draftField |
rows whose draft field is true | exclude_draft |
logicalDeletionField |
rows whose deletion field is not NULL | exclude_logical_deleted |
Scene 02
Declaring them in the recipe
In the course recipe the customer table already has two policies. sys_fields with draft equal True adds the boolean column underscore underscore is_draft and declares it as draftField. sys_fields also adds the column underscore underscore del_ts and declares it as logicalDeletionField. That happens by default, through the ldel option. Every table of the recipe that calls sys_fields has it. For this lesson we add a partition, copied from the example application. The attribute partition_state equal invc_state partitions the table on the column state. The value invc_state names two environment keys: current_invc_state and allowed_invc_state. Only the first partition attribute of a table counts.
Example & screen notes
Slide title "Declarations". Code, the line with partition_state highlighted as an addition:
def customer(pkg):
tbl = pkg.table('customer', pkey='id', name_long='Customer',
caption_field='account_name',
partition_state='invc_state') # addition
tbl.options().sys_fields(draft=True)
...
Bullets:
- sys_fields(draft=True) adds __is_draft and sets draftField='__is_draft'
- sys_fields() adds __del_ts and sets logicalDeletionField='__del_ts' (option ldel, default True)
- partition_state='invc_state': column state, environment keys current_invc_state and allowed_invc_state
Scene 03
Partition: current_
When the environment key current_invc_state has a true value, the compiler adds the condition state equals that value. The value is a parameter, read from the environment at execution. Look at the order of the conditions. Your where clause comes first. Then the partition condition, then the deletion condition, then the draft condition. Each condition is in its own parentheses, and they are joined with AND. An OR in your where clause cannot widen the partition.
Example & screen notes
Slide title "current_invc_state". Code:
customer = db.table('invc.customer')
names = lambda query: [row['account_name'] for row in query.fetch()]
with db.temp_env(current_invc_state='NSW'):
customer.query('$account_name').sqltext
# WHERE ("t0"."state" = %(env_current_invc_state)s)
# AND ("t0"."__del_ts" IS NULL) AND ("t0"."__is_draft" IS NOT TRUE)
names(customer.query('$account_name', order_by='$account_name'))
# ['Ada Pty']
Scene 04
Partition: allowed_ and no context
When current is not set and allowed_invc_state holds a list, the condition admits the listed states and also rows whose state is NULL. When both keys are set, current wins and allowed is ignored. When neither key has a true value, there is no partition condition, and the query returns rows of every state. A value of None, zero, an empty string or an empty list counts as not set. A missing context is not an error. The shape of the condition is chosen when the query compiles. A query compiled with current set keeps the condition state equals parameter. Executed later without the key, it compares with NULL and returns no rows.
Example & screen notes
Slide title "allowed_invc_state". Code:
with db.temp_env(allowed_invc_state=['NSW', 'VIC']):
customer.query('$account_name').sqltext
# WHERE ("t0"."state" IS NULL OR "t0"."state" IN(%(env_allowed_invc_state_0)s,
# %(env_allowed_invc_state_1)s)) ...
names(customer.query('$account_name', order_by='$account_name'))
# no partition key: ['Ada Pty', 'Bob Ltd'] no partition condition
Table:
| Environment | Condition |
|---|---|
current_<path> true |
$<field> = :env_current_<path> |
else allowed_<path> true |
$<field> IS NULL OR $<field> IN :env_allowed_<path> |
| neither true | none |
Scene 05
ignore_partition
ignore_partition equal True removes the partition condition for one query. The draft and deletion conditions stay. The option accepts only True or False; any other value raises ValueError. The same holds for exclude_draft.
Example & screen notes
Slide title "ignore_partition". Code:
with db.temp_env(current_invc_state='NSW'):
names(customer.query('$account_name', order_by='$account_name',
ignore_partition=True))
# ['Ada Pty', 'Bob Ltd']
Scene 06
Drafts
With a draft field declared, every SELECT adds the condition draft field IS NOT TRUE. Rows with false and rows with NULL stay visible. Cy Draft is hidden. exclude_draft equal False shows the drafts too. The option does not change any stored value.
Example & screen notes
Slide title "exclude_draft". Code:
names(customer.query('$account_name', order_by='$account_name'))
# ['Ada Pty', 'Bob Ltd']
names(customer.query('$account_name', order_by='$account_name', exclude_draft=False))
# ['Ada Pty', 'Bob Ltd', 'Cy Draft']
Scene 07
Logical deletion
A row is logically deleted when its deletion field is not NULL. In this example Bob Ltd has a timestamp in underscore underscore del_ts. By default the query adds del_ts IS NULL, and Bob disappears. exclude_logical_deleted equal False removes the condition. The value mark removes the condition and adds a result column named underscore isdeleted. It holds the deletion value itself, here a timestamp, or NULL. It is not converted to a boolean. In a query with distinct or group_by, mark adds no column. Any value other than True, False or mark raises ValueError.
Example & screen notes
Slide title "exclude_logical_deleted". Code, after Bob Ltd received a value in __del_ts:
names(customer.query('$account_name', order_by='$account_name'))
# ['Ada Pty']
names(customer.query('$account_name', order_by='$account_name',
exclude_logical_deleted=False))
# ['Ada Pty', 'Bob Ltd']
rows = customer.query('$account_name', order_by='$account_name',
exclude_logical_deleted='mark').fetch()
[(row['account_name'], row['_isdeleted'] is not None) for row in rows]
# [('Ada Pty', False), ('Bob Ltd', True)]
Bullets:
- True (default): adds __del_ts IS NULL
- False: no condition
- 'mark': no condition, adds the column _isdeleted with the deletion value
Scene 08
Spellings
Each policy option also accepts a camelCase spelling. Both spellings have the same effect. Use one of them per option. Passing both spellings of the same option raises ValueError, even when the values are equal.
Example & screen notes
Slide title "Two spellings". Table:
| Option | Other spelling |
|---|---|
exclude_draft |
excludeDraft |
exclude_logical_deleted |
excludeLogicalDeleted |
ignore_partition |
ignorePartition |
Code:
customer.query('$account_name', ignorePartition=True, ignore_partition=True)
# ValueError: Conflicting query options: ignorePartition and ignore_partition
Scene 09
Where policies do not apply
Policies apply to the root table of the SELECT only. A joined table is not filtered. The invoices of Bob Ltd still show his name, although Bob is logically deleted. Subqueries of formula columns include drafts and deleted rows and ignore the partition, unless their definition sets the options. Raw SQL through db dot execute applies no policy. Writes do not read policies either. An insert does not fill the partition column from the environment, and it can store a row outside the current partition. An update can change a draft row. Set the partition column in the record yourself. This release has no partition check on writes.
Example & screen notes
Slide title "Scope". Bullets:
- only the root table of the SELECT
- not joined tables, not formula subqueries by default
- not writes: insert, update, delete
- not raw SQL: db.execute(...)
Code:
invoice.query('$inv_number, @customer_id.account_name AS customer',
order_by='$inv_number').fetch()
# B1 and B2 still show 'Bob Ltd': the join is not filtered
db.execute('SELECT account_name FROM invc.invc_customer').fetchall()
# all three customers
with db.temp_env(current_invc_state='VIC'):
customer.insert({'account_name': 'Dee', 'state': 'NSW', 'customer_type_code': 'RET'})
# stored with state 'NSW'; not visible in this scope
Scene 10
Recap
To recap. The three policy attributes filter the root table of every SELECT. Three query options change them for one query, in one spelling at a time. Without partition context there is no partition filter, and writes and raw SQL are never filtered. The next lesson covers locking rows on read.
Example & screen notes
Slide title "Recap". Bullets:
- partition_<field>, draftField, logicalDeletionField filter the root table of every SELECT
- ignore_partition, exclude_draft, exclude_logical_deleted change them per query; one spelling each
- no partition context means no partition filter; writes and raw SQL are not filtered
Footer: "Next: lesson 19 — Locking on read".