# invalid_category
This rule checks if the rows of a categorical column in a table have the
right values. A right value is one in the set of allowable values for
the categorical column. For example, the `coll` column in the `samples`
table is a categorical column whose set of allowable values or
categories are `comp3h`, `comp8h`, `flowPr` etc. The following samples
table row would fail validation,
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"><span style="font-style: italic"> Invalid Dataset </span>
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃<span style="font-weight: bold"> coll </span>┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ flow │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
</pre>
The following samples table row would pass validation,
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"><span style="font-style: italic"> Valid Dataset </span>
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃<span style="font-weight: bold"> coll </span>┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ flowPr │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
</pre>
For non-mandatory categorical columns, values representing missing are
also allowed. Assuming that `coll` is not mandatory, the following table
should pass validation,
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"><span style="font-style: italic"> Valid Dataset </span>
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃<span style="font-weight: bold"> coll </span>┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ NA │
│ │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
</pre>
The second row contains an empty string.
## Error report
The error report will have the following fields
- **errorType**: invalid_category
- **tableName**: The name of the table whose row has the invalid
category
- **columnName** The name of the column with the invalid category
- **rowNumber**: The index of the table row with the error
- **row** The row in the data that failed this validation rule
- **invalidValue**: The invalid category value
- **validationRuleFields**: The ODM data dictionary rule fields violated
by this row
- **message**: Invalid category \<invalidValue\> found in row
\<rowIndex\> for column \<columnName\> in table \<tableName\>
Example
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ </span><span style="color: #008000; text-decoration-color: #008000">'errors'</span>: <span style="font-weight: bold">[</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'errorType'</span>: <span style="color: #008000; text-decoration-color: #008000">'invalid_category'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'tableName'</span>: <span style="color: #008000; text-decoration-color: #008000">'samples'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'columnName'</span>: <span style="color: #008000; text-decoration-color: #008000">'coll'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'rowNumber'</span>: <span style="color: #008080; text-decoration-color: #008080; font-weight: bold">1</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'row'</span>: <span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'coll'</span>: <span style="color: #008000; text-decoration-color: #008000">'flow'</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ </span><span style="font-weight: bold">}</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'invalidValue'</span>: <span style="color: #008000; text-decoration-color: #008000">'flow'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'validationRuleFields'</span>: <span style="font-weight: bold">[</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'partID'</span>: <span style="color: #008000; text-decoration-color: #008000">'coll'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'samples'</span>: <span style="color: #008000; text-decoration-color: #008000">'header'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'dataType'</span>: <span style="color: #008000; text-decoration-color: #008000">'categorical'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'mmaSet'</span>: <span style="color: #008000; text-decoration-color: #008000">'collectCat'</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ </span><span style="font-weight: bold">}</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'partID'</span>: <span style="color: #008000; text-decoration-color: #008000">'comp3h'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'setID'</span>: <span style="color: #008000; text-decoration-color: #008000">'collectCat'</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ </span><span style="font-weight: bold">}</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'partID'</span>: <span style="color: #008000; text-decoration-color: #008000">'comp8h'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'setID'</span>: <span style="color: #008000; text-decoration-color: #008000">'collectCat'</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ </span><span style="font-weight: bold">}</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'partID'</span>: <span style="color: #008000; text-decoration-color: #008000">'flowPr'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'setID'</span>: <span style="color: #008000; text-decoration-color: #008000">'collectCat'</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ </span><span style="font-weight: bold">}</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ </span><span style="font-weight: bold">]</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'message'</span>: <span style="color: #008000; text-decoration-color: #008000">'invalid_category rule violated in table samples, column coll, row(s) 1: Invalid category "flow"'</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ </span><span style="font-weight: bold">}</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ </span><span style="font-weight: bold">]</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ </span><span style="color: #008000; text-decoration-color: #008000">'warnings'</span>: <span style="font-weight: bold">[]</span>
<span style="font-weight: bold">}</span>
</pre>
## Rule metadata
The metadata for this rule is contained in two sheets:
1. The parts sheet which has information on whether a column is
categorical and if it is, information on how to find the categories
that are part of it
2. The sets sheet which contains information on the categories
The steps to use this meatdata are:
1. [Get all the columns that are part of the
table](../specs/odm-how-tos.md#how-to-get-the-columns-names-for-a-table)
2. Filter the columns to only include those that are categorical.
Categorical columns have a `dataType` value of `categorical`.
3. For each categorical column, identify the name of the set that has
the categories for that column. The set name is stored in the
`mmaSet` column.
4. Use the sets sheet and the value of the `mmaSet` column to identify
the categories. For each categorical column, filter the sets sheet
to only include those rows whose `setID` column value is the same as
the `mmaSet` value.
5. In the filtered rows, the `partID` column contains the category
values.
Example
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"><span style="font-style: italic"> Parts v2 </span>
┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃<span style="font-weight: bold"> partID </span>┃<span style="font-weight: bold"> partType </span>┃<span style="font-weight: bold"> samples </span>┃<span style="font-weight: bold"> sites </span>┃<span style="font-weight: bold"> measures </span>┃<span style="font-weight: bold"> dataType </span>┃<span style="font-weight: bold"> mmaSet </span>┃<span style="font-weight: bold"> status </span>┃<span style="font-weight: bold"> firstRele… </span>┃<span style="font-weight: bold"> lastUpda… </span>┃
┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ samples │ tables │ NA │ NA │ NA │ NA │ NA │ active │ 1.0.0 │ 2.0.0 │
│ sites │ tables │ NA │ NA │ NA │ NA │ NA │ active │ 1.0.0 │ 2.0.0 │
│ measures │ tables │ NA │ NA │ NA │ NA │ NA │ active │ 1.0.0 │ 2.0.0 │
│ coll │ attribut… │ header │ NA │ NA │ categori… │ collectC… │ active │ 1.0.0 │ 2.0.0 │
│ comp3h │ categori… │ input │ NA │ NA │ varchar │ NA │ active │ 1.0.0 │ 2.0.0 │
│ comp8h │ categori… │ input │ NA │ NA │ varchar │ NA │ active │ 1.0.0 │ 2.0.0 │
│ flowPr │ categori… │ input │ NA │ NA │ varchar │ NA │ active │ 1.0.0 │ 2.0.0 │
│ siteType… │ attribut… │ NA │ fK │ NA │ categori… │ siteType… │ active │ 1.0.0 │ 2.0.0 │
│ wwtpMuC │ categori… │ NA │ input │ NA │ varchar │ NA │ active │ 1.0.0 │ 2.0.0 │
│ wwtpMuS │ categori… │ NA │ input │ NA │ varchar │ NA │ active │ 1.0.0 │ 2.0.0 │
│ someOldC… │ categori… │ NA │ input │ NA │ varchar │ NA │ deprecia… │ 1.0.0 │ 2.0.0 │
│ tp24s │ categori… │ NA │ NA │ NA │ NA │ NA │ deprecia… │ 1.0.0 │ 2.0.0 │
│ measureID │ attribut… │ NA │ NA │ fK │ varchar │ NA │ active │ 1.0.0 │ 2.0.0 │
│ cod │ categori… │ NA │ NA │ input │ varchar │ NA │ active │ 1.0.0 │ 2.0.0 │
│ NA │ missingn… │ NA │ NA │ NA │ varchar │ NA │ active │ 1.0.0 │ 2.0.0 │
└───────────┴───────────┴─────────┴───────┴──────────┴───────────┴───────────┴───────────┴────────────┴───────────┘
</pre>
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"><span style="font-style: italic"> Sets </span>
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃<span style="font-weight: bold"> setID </span>┃<span style="font-weight: bold"> partID </span>┃<span style="font-weight: bold"> firstReleased </span>┃<span style="font-weight: bold"> lastUpdated </span>┃<span style="font-weight: bold"> status </span>┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ collectCat │ comp3h │ 1.0.0 │ 2.0.0 │ active │
│ collectCat │ comp8h │ 1.0.0 │ 2.0.0 │ active │
│ collectCat │ flowPr │ 1.0.0 │ 2.0.0 │ active │
└────────────────────────┴────────────────┴─────────────────────────────┴─────────────────────────┴───────────────┘
</pre>
Here, the name of the categorical column is `coll` and its a column in
the `samples` table. The set name is `collectCat` which is used to
identify the category values in the sets sheet which are `comp3h`,
`comp8h`, and `flowPr`.
## Cerberus Schema
The generated cerberus object for the example above is shown below,
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ </span><span style="color: #008000; text-decoration-color: #008000">'schemaVersion'</span>: <span style="color: #008000; text-decoration-color: #008000">'2.0.0'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ </span><span style="color: #008000; text-decoration-color: #008000">'schema'</span>: <span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ </span><span style="color: #008000; text-decoration-color: #008000">'samples'</span>: <span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'type'</span>: <span style="color: #008000; text-decoration-color: #008000">'list'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'schema'</span>: <span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'type'</span>: <span style="color: #008000; text-decoration-color: #008000">'dict'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'schema'</span>: <span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'coll'</span>: <span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'anyof'</span>: <span style="font-weight: bold">[</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'allowed'</span>: <span style="font-weight: bold">[</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'comp3h'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'comp8h'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'flowPr'</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ </span><span style="font-weight: bold">]</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'empty'</span>: <span style="color: #00ff00; text-decoration-color: #00ff00; font-style: italic">True</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ </span><span style="font-weight: bold">}</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="font-weight: bold">]</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'meta'</span>: <span style="font-weight: bold">[</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'ruleID'</span>: <span style="color: #008000; text-decoration-color: #008000">'invalid_category'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'meta'</span>: <span style="font-weight: bold">[</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'partID'</span>: <span style="color: #008000; text-decoration-color: #008000">'coll'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'samples'</span>: <span style="color: #008000; text-decoration-color: #008000">'header'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'dataType'</span>: <span style="color: #008000; text-decoration-color: #008000">'categorical'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'mmaSet'</span>: <span style="color: #008000; text-decoration-color: #008000">'collectCat'</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="font-weight: bold">}</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'partID'</span>: <span style="color: #008000; text-decoration-color: #008000">'comp3h'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'setID'</span>: <span style="color: #008000; text-decoration-color: #008000">'collectCat'</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="font-weight: bold">}</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'partID'</span>: <span style="color: #008000; text-decoration-color: #008000">'comp8h'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'setID'</span>: <span style="color: #008000; text-decoration-color: #008000">'collectCat'</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="font-weight: bold">}</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'partID'</span>: <span style="color: #008000; text-decoration-color: #008000">'flowPr'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'setID'</span>: <span style="color: #008000; text-decoration-color: #008000">'collectCat'</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="font-weight: bold">}</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ </span><span style="font-weight: bold">]</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ </span><span style="font-weight: bold">}</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="font-weight: bold">]</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ </span><span style="font-weight: bold">}</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ </span><span style="font-weight: bold">}</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'meta'</span>: <span style="font-weight: bold">[</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'partID'</span>: <span style="color: #008000; text-decoration-color: #008000">'samples'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'partType'</span>: <span style="color: #008000; text-decoration-color: #008000">'tables'</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ </span><span style="font-weight: bold">}</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ </span><span style="font-weight: bold">]</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ </span><span style="font-weight: bold">}</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ </span><span style="font-weight: bold">}</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ </span><span style="font-weight: bold">}</span>
<span style="font-weight: bold">}</span>
</pre>
The metadata for this rule should include the following rows from the
ODM dictionary:
- The part definition for the column in the table
- The sets definition for each category that forms the category set for
the categorical column
## Version 1
Generating the cerberus schema for version 1 requires the following
information:
1. The columns that are part of the version 1 table
2. Whether the column is categorical and
3. If it is, the list of allowed categories
Information on point 1 can be found [here](./README.md#version-1).
To check whether a version 1 column is categorical, we can use the
`version1Location` column. If the column has a value of
`variableCategories` then the part was a category in version 1. We can
then look at the value of the `version1Category` column to see the what
the category value was in version 1. For example, in the ODM parts
snippet below,
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"><span style="font-style: italic"> Parts v1 </span>
┏━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┓
┃<span style="font-weight: bold"> part… </span>┃<span style="font-weight: bold"> part… </span>┃<span style="font-weight: bold"> samp… </span>┃<span style="font-weight: bold"> sites </span>┃<span style="font-weight: bold"> meas… </span>┃<span style="font-weight: bold"> data… </span>┃<span style="font-weight: bold"> mmaS… </span>┃<span style="font-weight: bold"> vers… </span>┃<span style="font-weight: bold"> vers… </span>┃<span style="font-weight: bold"> vers… </span>┃<span style="font-weight: bold"> vers… </span>┃<span style="font-weight: bold"> status </span>┃<span style="font-weight: bold"> firs… </span>┃<span style="font-weight: bold"> lastU… </span>┃
┡━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━┩
│ samp… │ tabl… │ NA │ NA │ NA │ NA │ NA │ tabl… │ Samp… │ NA │ NA │ active │ 1.0.0 │ 2.0.0 │
│ sites │ tabl… │ NA │ NA │ NA │ NA │ NA │ tabl… │ Site │ NA │ NA │ active │ 1.0.0 │ 2.0.0 │
│ meas… │ tabl… │ NA │ NA │ NA │ NA │ NA │ tabl… │ WWMe… │ NA │ NA │ active │ 1.0.0 │ 2.0.0 │
│ coll │ attr… │ head… │ NA │ NA │ cate… │ coll… │ vari… │ Samp… │ Coll… │ NA │ active │ 1.0.0 │ 2.0.0 │
│ comp… │ cate… │ input │ NA │ NA │ varc… │ NA │ vari… │ Samp… │ Coll… │ Comp… │ active │ 1.0.0 │ 2.0.0 │
│ comp… │ cate… │ input │ NA │ NA │ varc… │ NA │ vari… │ Samp… │ Coll… │ Comp… │ active │ 1.0.0 │ 2.0.0 │
│ flow… │ cate… │ input │ NA │ NA │ varc… │ NA │ vari… │ Samp… │ Coll… │ Flow… │ active │ 1.0.0 │ 2.0.0 │
│ site… │ attr… │ NA │ fK │ NA │ cate… │ site… │ vari… │ Site │ type │ NA │ active │ 1.0.0 │ 2.0.0 │
│ wwtp… │ cate… │ NA │ input │ NA │ varc… │ NA │ vari… │ Site │ type │ wwtp… │ active │ 1.0.0 │ 2.0.0 │
│ wwtp… │ cate… │ NA │ input │ NA │ varc… │ NA │ vari… │ Site │ type │ wwtp… │ active │ 1.0.0 │ 2.0.0 │
│ some… │ cate… │ NA │ input │ NA │ varc… │ NA │ vari… │ Site │ type │ some… │ depre… │ 1.0.0 │ 2.0.0 │
│ tp24s │ cate… │ NA │ NA │ NA │ NA │ NA │ vari… │ Site │ Samp… │ cpTP… │ depre… │ 1.0.0 │ 2.0.0 │
│ meas… │ attr… │ NA │ NA │ fK │ varc… │ NA │ vari… │ WWMe… │ type │ NA │ active │ 1.0.0 │ 2.0.0 │
│ cod │ cate… │ NA │ NA │ input │ varc… │ NA │ vari… │ WWMe… │ type │ wqCO… │ active │ 1.0.0 │ 2.0.0 │
│ │ │ │ │ │ │ │ │ Site… │ │ wwCOD │ │ │ │
│ NA │ miss… │ NA │ NA │ NA │ varc… │ NA │ NA │ NA │ NA │ NA │ active │ 1.0.0 │ 2.0.0 │
└───────┴───────┴───────┴───────┴───────┴───────┴───────┴───────┴───────┴───────┴───────┴────────┴───────┴────────┘
</pre>
In the snippet above, for a version 1 variable, we check whether the
`version1Category` column has a value. If it does then it is
categorical. We can then retreive the categories by looking at all the
unique `version1Category` values for that variable. Keep in mind, that
there can be multiple categories encoded in a cell, with each value
seperated by a semi-colon.
The version 1 variable `Collection` column has four categories,
`Comp3h`, `Comp8h`, `FlowPR`, `FlowRatePr`.
The corresponding cerberus schema for version 1 would be,
<pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ </span><span style="color: #008000; text-decoration-color: #008000">'schemaVersion'</span>: <span style="color: #008000; text-decoration-color: #008000">'1.0.0'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ </span><span style="color: #008000; text-decoration-color: #008000">'schema'</span>: <span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ </span><span style="color: #008000; text-decoration-color: #008000">'Sample'</span>: <span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'type'</span>: <span style="color: #008000; text-decoration-color: #008000">'list'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'schema'</span>: <span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'type'</span>: <span style="color: #008000; text-decoration-color: #008000">'dict'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'schema'</span>: <span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'Collection'</span>: <span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'anyof'</span>: <span style="font-weight: bold">[</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'allowed'</span>: <span style="font-weight: bold">[</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'Comp3h'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'Comp8h'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'FlowPr'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'FlowRatePr'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'other'</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ </span><span style="font-weight: bold">]</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'empty'</span>: <span style="color: #00ff00; text-decoration-color: #00ff00; font-style: italic">True</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ </span><span style="font-weight: bold">}</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="font-weight: bold">]</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'meta'</span>: <span style="font-weight: bold">[</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'ruleID'</span>: <span style="color: #008000; text-decoration-color: #008000">'invalid_category'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'meta'</span>: <span style="font-weight: bold">[</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'partID'</span>: <span style="color: #008000; text-decoration-color: #008000">'coll'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'dataType'</span>: <span style="color: #008000; text-decoration-color: #008000">'categorical'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Location'</span>: <span style="color: #008000; text-decoration-color: #008000">'variables'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Table'</span>: <span style="color: #008000; text-decoration-color: #008000">'Sample'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Variable'</span>: <span style="color: #008000; text-decoration-color: #008000">'Collection'</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="font-weight: bold">}</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'partID'</span>: <span style="color: #008000; text-decoration-color: #008000">'comp3h'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Location'</span>: <span style="color: #008000; text-decoration-color: #008000">'variableCategories'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Table'</span>: <span style="color: #008000; text-decoration-color: #008000">'Sample'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Variable'</span>: <span style="color: #008000; text-decoration-color: #008000">'Collection'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Category'</span>: <span style="color: #008000; text-decoration-color: #008000">'Comp3h'</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="font-weight: bold">}</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'partID'</span>: <span style="color: #008000; text-decoration-color: #008000">'comp8h'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Location'</span>: <span style="color: #008000; text-decoration-color: #008000">'variableCategories'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Table'</span>: <span style="color: #008000; text-decoration-color: #008000">'Sample'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Variable'</span>: <span style="color: #008000; text-decoration-color: #008000">'Collection'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Category'</span>: <span style="color: #008000; text-decoration-color: #008000">'Comp8h'</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="font-weight: bold">}</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'partID'</span>: <span style="color: #008000; text-decoration-color: #008000">'flowPr'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Location'</span>: <span style="color: #008000; text-decoration-color: #008000">'variableCategories'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Table'</span>: <span style="color: #008000; text-decoration-color: #008000">'Sample'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Variable'</span>: <span style="color: #008000; text-decoration-color: #008000">'Collection'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Category'</span>: <span style="color: #008000; text-decoration-color: #008000">'FlowPr;FlowRatePr'</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="font-weight: bold">}</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ </span><span style="font-weight: bold">]</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ </span><span style="font-weight: bold">}</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="font-weight: bold">]</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ </span><span style="font-weight: bold">}</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ </span><span style="font-weight: bold">}</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'meta'</span>: <span style="font-weight: bold">[</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'partID'</span>: <span style="color: #008000; text-decoration-color: #008000">'samples'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'partType'</span>: <span style="color: #008000; text-decoration-color: #008000">'tables'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Location'</span>: <span style="color: #008000; text-decoration-color: #008000">'tables'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Table'</span>: <span style="color: #008000; text-decoration-color: #008000">'Sample'</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ </span><span style="font-weight: bold">}</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ </span><span style="font-weight: bold">]</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ </span><span style="font-weight: bold">}</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ </span><span style="font-weight: bold">}</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ </span><span style="color: #008000; text-decoration-color: #008000">'Site'</span>: <span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'type'</span>: <span style="color: #008000; text-decoration-color: #008000">'list'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'schema'</span>: <span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'type'</span>: <span style="color: #008000; text-decoration-color: #008000">'dict'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'schema'</span>: <span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'type'</span>: <span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'anyof'</span>: <span style="font-weight: bold">[</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'allowed'</span>: <span style="font-weight: bold">[</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'other'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'someOldCat1'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'wwtpMuC'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'wwtpMuS'</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ </span><span style="font-weight: bold">]</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'empty'</span>: <span style="color: #00ff00; text-decoration-color: #00ff00; font-style: italic">True</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ </span><span style="font-weight: bold">}</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="font-weight: bold">]</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'meta'</span>: <span style="font-weight: bold">[</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'ruleID'</span>: <span style="color: #008000; text-decoration-color: #008000">'invalid_category'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'meta'</span>: <span style="font-weight: bold">[</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'partID'</span>: <span style="color: #008000; text-decoration-color: #008000">'siteTypeID'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'dataType'</span>: <span style="color: #008000; text-decoration-color: #008000">'categorical'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Location'</span>: <span style="color: #008000; text-decoration-color: #008000">'variables'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Table'</span>: <span style="color: #008000; text-decoration-color: #008000">'Site'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Variable'</span>: <span style="color: #008000; text-decoration-color: #008000">'type'</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="font-weight: bold">}</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'partID'</span>: <span style="color: #008000; text-decoration-color: #008000">'wwtpMuC'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Category'</span>: <span style="color: #008000; text-decoration-color: #008000">'wwtpMuC'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Location'</span>: <span style="color: #008000; text-decoration-color: #008000">'variableCategories'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Table'</span>: <span style="color: #008000; text-decoration-color: #008000">'Site'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Variable'</span>: <span style="color: #008000; text-decoration-color: #008000">'type'</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="font-weight: bold">}</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'partID'</span>: <span style="color: #008000; text-decoration-color: #008000">'wwtpMuS'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Category'</span>: <span style="color: #008000; text-decoration-color: #008000">'wwtpMuS'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Location'</span>: <span style="color: #008000; text-decoration-color: #008000">'variableCategories'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Table'</span>: <span style="color: #008000; text-decoration-color: #008000">'Site'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Variable'</span>: <span style="color: #008000; text-decoration-color: #008000">'type'</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="font-weight: bold">}</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'partID'</span>: <span style="color: #008000; text-decoration-color: #008000">'someOldCat'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Category'</span>: <span style="color: #008000; text-decoration-color: #008000">'someOldCat1'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Location'</span>: <span style="color: #008000; text-decoration-color: #008000">'variableCategories'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Table'</span>: <span style="color: #008000; text-decoration-color: #008000">'Site'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Variable'</span>: <span style="color: #008000; text-decoration-color: #008000">'type'</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="font-weight: bold">}</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ </span><span style="font-weight: bold">]</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ </span><span style="font-weight: bold">}</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="font-weight: bold">]</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ </span><span style="font-weight: bold">}</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ </span><span style="font-weight: bold">}</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'meta'</span>: <span style="font-weight: bold">[</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'partID'</span>: <span style="color: #008000; text-decoration-color: #008000">'sites'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'partType'</span>: <span style="color: #008000; text-decoration-color: #008000">'tables'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Location'</span>: <span style="color: #008000; text-decoration-color: #008000">'tables'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Table'</span>: <span style="color: #008000; text-decoration-color: #008000">'Site'</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ </span><span style="font-weight: bold">}</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ </span><span style="font-weight: bold">]</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ </span><span style="font-weight: bold">}</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ </span><span style="font-weight: bold">}</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ </span><span style="color: #008000; text-decoration-color: #008000">'WWMeasure'</span>: <span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'type'</span>: <span style="color: #008000; text-decoration-color: #008000">'list'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'schema'</span>: <span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'type'</span>: <span style="color: #008000; text-decoration-color: #008000">'dict'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'schema'</span>: <span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'type'</span>: <span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'anyof'</span>: <span style="font-weight: bold">[</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'allowed'</span>: <span style="font-weight: bold">[</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'other'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'wqCOD'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'wwCOD'</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ </span><span style="font-weight: bold">]</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'empty'</span>: <span style="color: #00ff00; text-decoration-color: #00ff00; font-style: italic">True</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ </span><span style="font-weight: bold">}</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="font-weight: bold">]</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'meta'</span>: <span style="font-weight: bold">[</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'ruleID'</span>: <span style="color: #008000; text-decoration-color: #008000">'invalid_category'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'meta'</span>: <span style="font-weight: bold">[</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'dataType'</span>: <span style="color: #008000; text-decoration-color: #008000">'varchar'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'partID'</span>: <span style="color: #008000; text-decoration-color: #008000">'measureID'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Location'</span>: <span style="color: #008000; text-decoration-color: #008000">'variables'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Table'</span>: <span style="color: #008000; text-decoration-color: #008000">'WWMeasure'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Variable'</span>: <span style="color: #008000; text-decoration-color: #008000">'type'</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="font-weight: bold">}</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'partID'</span>: <span style="color: #008000; text-decoration-color: #008000">'cod'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Category'</span>: <span style="color: #008000; text-decoration-color: #008000">'wqCOD; wwCOD'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Location'</span>: <span style="color: #008000; text-decoration-color: #008000">'variableCategories'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Table'</span>: <span style="color: #008000; text-decoration-color: #008000">'WWMeasure; SiteMeasure'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Variable'</span>: <span style="color: #008000; text-decoration-color: #008000">'type'</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ │ </span><span style="font-weight: bold">}</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ │ </span><span style="font-weight: bold">]</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ │ </span><span style="font-weight: bold">}</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="font-weight: bold">]</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ </span><span style="font-weight: bold">}</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ </span><span style="font-weight: bold">}</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'meta'</span>: <span style="font-weight: bold">[</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ </span><span style="font-weight: bold">{</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'partID'</span>: <span style="color: #008000; text-decoration-color: #008000">'measures'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'partType'</span>: <span style="color: #008000; text-decoration-color: #008000">'tables'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Location'</span>: <span style="color: #008000; text-decoration-color: #008000">'tables'</span>,
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ │ </span><span style="color: #008000; text-decoration-color: #008000">'version1Table'</span>: <span style="color: #008000; text-decoration-color: #008000">'WWMeasure'</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ │ </span><span style="font-weight: bold">}</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ │ </span><span style="font-weight: bold">]</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ │ </span><span style="font-weight: bold">}</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ │ </span><span style="font-weight: bold">}</span>
<span style="color: #7fbf7f; text-decoration-color: #7fbf7f">│ </span><span style="font-weight: bold">}</span>
<span style="font-weight: bold">}</span>
</pre>
The meta field for the rule should include the same part rows and fields
as version 2 with the following additions,
- The `version1Location`, `version1Table`, and version1Variable\`
columns added to all the rows
- The `version1Category` column added to the category parts