Scalars and columns of any element type.
Value(self, arg)
Base class for a data generating expression having a known type.
| Name | Description |
|---|---|
| asc | Sort an expression ascending. |
| cast | Cast expression to indicated data type. |
| coalesce | Return the first non-null value from args. |
| collect | Aggregate this expression’s elements into an array. |
| identical_to | Return whether this expression is identical to other. |
| isin | Check whether this expression’s values are in values. |
| isnull | Return whether this expression is NULL. |
| name | Rename an expression to name. |
| notnull | Return whether this expression is not NULL. |
| nullif | Set values to null if they equal the values null_if_expr. |
| try_cast | Try cast expression to indicated data type. |
asc()
Sort an expression ascending.
cast(target_type)
Cast expression to indicated data type.
Similar to pandas.Series.astype.
| Name | Type | Description | Default |
|---|---|---|---|
target_type |
Any | Type to cast to. Anything accepted by ibis.dtype() |
required |
| Type | Description |
|---|---|
| Value | Casted expression |
>>> import ibis
>>> ibis.options.interactive = True
>>> x = letsql.examples.penguins.fetch()["bill_depth_mm"]
>>> x┏━━━━━━━━━━━━━━━┓ ┃ bill_depth_mm ┃ ┡━━━━━━━━━━━━━━━┩ │ float64 │ ├───────────────┤ │ 18.7 │ │ 17.4 │ │ 18.0 │ │ NULL │ │ 19.3 │ │ 20.6 │ │ 17.8 │ │ 19.6 │ │ 18.1 │ │ 20.2 │ │ … │ └───────────────┘
python’s built-in types can be used
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ Cast(bill_depth_mm, int64) ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │ int64 │ ├────────────────────────────┤ │ 18 │ │ 17 │ │ 18 │ │ NULL │ │ 19 │ │ 20 │ │ 17 │ │ 19 │ │ 18 │ │ 20 │ │ … │ └────────────────────────────┘
or string names
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ Cast(bill_depth_mm, int8) ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │ int8 │ ├───────────────────────────┤ │ 18 │ │ 17 │ │ 18 │ │ NULL │ │ 19 │ │ 20 │ │ 17 │ │ 19 │ │ 18 │ │ 20 │ │ … │ └───────────────────────────┘
If you make an illegal cast, you won’t know until the backend actually executes it. Consider .try_cast().
coalesce(*args)
Return the first non-null value from args.
| Name | Type | Description | Default |
|---|---|---|---|
args |
Value | Arguments from which to choose the first non-null value | () |
| Type | Description |
|---|---|
| Value | Coalesced expression |
collect(where=None)
Aggregate this expression’s elements into an array.
This function is called array_agg, list_agg, or list in other systems.
| Name | Type | Description | Default |
|---|---|---|---|
where |
ir.BooleanValue | None | Filter to apply before aggregation | None |
| Type | Description |
|---|---|
| ArrayScalar | Collected array |
Basic collect usage
>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"key": list("aaabb"), "value": [1, 2, 3, 4, 5]})
>>> t┏━━━━━━━━┳━━━━━━━┓ ┃ key ┃ value ┃ ┡━━━━━━━━╇━━━━━━━┩ │ string │ int64 │ ├────────┼───────┤ │ a │ 1 │ │ a │ 2 │ │ a │ 3 │ │ b │ 4 │ │ b │ 5 │ └────────┴───────┘
Collect elements per group
┏━━━━━━━━┳━━━━━━━━━━━━━━━━┓ ┃ key ┃ v ┃ ┡━━━━━━━━╇━━━━━━━━━━━━━━━━┩ │ string │ array<int64> │ ├────────┼────────────────┤ │ a │ [1, 2, ... +1] │ │ b │ [4, 5] │ └────────┴────────────────┘
Collect elements per group using a filter
identical_to(other)
Return whether this expression is identical to other.
Corresponds to IS NOT DISTINCT FROM in SQL.
| Name | Type | Description | Default |
|---|---|---|---|
other |
Value | Expression to compare to | required |
| Type | Description |
|---|---|
| BooleanValue | Whether this expression is not distinct from other |
isin(values)
Check whether this expression’s values are in values.
NULL values are propagated in the output. See examples for details.
| Name | Type | Description | Default |
|---|---|---|---|
values |
Value | Sequence[Value] | Values or expression to check for membership | required |
| Type | Description |
|---|---|
| BooleanValue | Expression indicating membership |
>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"a": [1, 2, 3], "b": [2, 3, 4]})
>>> t┏━━━━━━━┳━━━━━━━┓ ┃ a ┃ b ┃ ┡━━━━━━━╇━━━━━━━┩ │ int64 │ int64 │ ├───────┼───────┤ │ 1 │ 2 │ │ 2 │ 3 │ │ 3 │ 4 │ └───────┴───────┘
Check against a literal sequence of values
┏━━━━━━━━━━━━━┓ ┃ InValues(a) ┃ ┡━━━━━━━━━━━━━┩ │ boolean │ ├─────────────┤ │ True │ │ True │ │ False │ └─────────────┘
Check against a derived expression
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ /home/runner/.cache/pypoetry/virtualenvs/letsql-uJiiVu6z-py3.10/lib/python3.10/site-packages/ibi │ │ s/expr/types/core.py:99 in __rich_console__ │ │ │ │ 96 │ │ │ │ 97 │ │ try: │ │ 98 │ │ │ if opts.interactive: │ │ ❱ 99 │ │ │ │ rich_object = to_rich(self, console_width=console_width) │ │ 100 │ │ │ else: │ │ 101 │ │ │ │ rich_object = Text(self._noninteractive_repr()) │ │ 102 │ │ except Exception as e: │ │ │ │ /home/runner/.cache/pypoetry/virtualenvs/letsql-uJiiVu6z-py3.10/lib/python3.10/site-packages/ibi │ │ s/expr/types/pretty.py:271 in to_rich │ │ │ │ 268 │ │ │ expr, max_length=max_length, max_string=max_string, max_depth=max_depth │ │ 269 │ │ ) │ │ 270 │ else: │ │ ❱ 271 │ │ return _to_rich_table( │ │ 272 │ │ │ expr, │ │ 273 │ │ │ max_rows=max_rows, │ │ 274 │ │ │ max_columns=max_columns, │ │ │ │ /home/runner/.cache/pypoetry/virtualenvs/letsql-uJiiVu6z-py3.10/lib/python3.10/site-packages/ibi │ │ s/expr/types/pretty.py:342 in _to_rich_table │ │ │ │ 339 │ │ if orig_ncols > len(computed_cols): │ │ 340 │ │ │ table = table.select(*computed_cols) │ │ 341 │ │ │ ❱ 342 │ result = table.limit(max_rows + 1).to_pyarrow() │ │ 343 │ # Now format the columns in order, stopping if the console width would │ │ 344 │ # be exceeded. │ │ 345 │ col_info = [] │ │ │ │ /home/runner/.cache/pypoetry/virtualenvs/letsql-uJiiVu6z-py3.10/lib/python3.10/site-packages/too │ │ lz/functoolz.py:304 in __call__ │ │ │ │ 301 │ │ │ 302 │ def __call__(self, *args, **kwargs): │ │ 303 │ │ try: │ │ ❱ 304 │ │ │ return self._partial(*args, **kwargs) │ │ 305 │ │ except TypeError as exc: │ │ 306 │ │ │ if self._should_curry(args, kwargs, exc): │ │ 307 │ │ │ │ return self.bind(*args, **kwargs) │ │ │ │ /home/runner/work/letsql/letsql/python/letsql/backends/let/hotfix.py:196 in letsql_invoke │ │ │ │ 193 │ │ if dt not in con._sources.sources: │ │ 194 │ │ │ con.register(dt.to_expr(), dt.name) │ │ 195 │ method = getattr(con, f"{_methodname}") │ │ ❱ 196 │ return method(self, *args, **kwargs) │ │ 197 │ │ 198 │ │ 199 for typ, methodnames in ( │ │ │ │ /home/runner/work/letsql/letsql/python/letsql/backends/let/__init__.py:233 in to_pyarrow │ │ │ │ 230 │ │ │ 231 │ def to_pyarrow(self, expr: ir.Expr, **kwargs: Any) -> pa.Table: │ │ 232 │ │ backend, expr = self._get_backend_and_expr(expr) │ │ ❱ 233 │ │ return backend.to_pyarrow(expr.unbind(), **kwargs) │ │ 234 │ │ │ 235 │ def to_pyarrow_batches( │ │ 236 │ │ self, │ │ │ │ /home/runner/work/letsql/letsql/python/letsql/backends/datafusion/__init__.py:727 in to_pyarrow │ │ │ │ 724 │ │ │ 725 │ def to_pyarrow(self, expr: ir.Expr, **kwargs: Any) -> pa.Table: │ │ 726 │ │ batch_reader = self._to_pyarrow_batches(expr, **kwargs) │ │ ❱ 727 │ │ arrow_table = batch_reader.read_all() │ │ 728 │ │ return expr.__pyarrow_result__(arrow_table) │ │ 729 │ │ │ 730 │ def execute(self, expr: ir.Expr, **kwargs: Any): │ │ │ │ in pyarrow.lib.RecordBatchReader.read_all:757 │ │ │ │ in pyarrow.lib.check_status:88 │ │ │ │ /home/runner/work/letsql/letsql/python/letsql/backends/datafusion/__init__.py:717 in make_gen │ │ │ │ 714 │ │ │ │ │ .to_struct_array() │ │ 715 │ │ │ │ │ .cast(struct_schema) │ │ 716 │ │ │ │ ) │ │ ❱ 717 │ │ │ │ for batch in frame.execute_stream() │ │ 718 │ │ │ ) │ │ 719 │ │ │ │ 720 │ │ return pa.ipc.RecordBatchReader.from_batches( │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ Exception: This feature is not implemented: Physical plan does not support logical expression InSubquery(InSubquery { expr: Column(Column { relation: Some(Bare { table: "t0" }), name: "a" }), subquery: <subquery>, negated: false })
Exception: This feature is not implemented: Physical plan does not support logical expression InSubquery(InSubquery { expr: Column(Column { relation: Some(Bare { table: "t0" }), name: "a" }), subquery: <subquery>, negated: false })Exception: This feature is not implemented: Physical plan does not support logical expression InSubquery(InSubquery { expr: Column(Column { relation: Some(Bare { table: "t0" }), name: "a" }), subquery: <subquery>, negated: false })Check against a column from a different table
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ /home/runner/.cache/pypoetry/virtualenvs/letsql-uJiiVu6z-py3.10/lib/python3.10/site-packages/ibi │ │ s/expr/types/core.py:99 in __rich_console__ │ │ │ │ 96 │ │ │ │ 97 │ │ try: │ │ 98 │ │ │ if opts.interactive: │ │ ❱ 99 │ │ │ │ rich_object = to_rich(self, console_width=console_width) │ │ 100 │ │ │ else: │ │ 101 │ │ │ │ rich_object = Text(self._noninteractive_repr()) │ │ 102 │ │ except Exception as e: │ │ │ │ /home/runner/.cache/pypoetry/virtualenvs/letsql-uJiiVu6z-py3.10/lib/python3.10/site-packages/ibi │ │ s/expr/types/pretty.py:271 in to_rich │ │ │ │ 268 │ │ │ expr, max_length=max_length, max_string=max_string, max_depth=max_depth │ │ 269 │ │ ) │ │ 270 │ else: │ │ ❱ 271 │ │ return _to_rich_table( │ │ 272 │ │ │ expr, │ │ 273 │ │ │ max_rows=max_rows, │ │ 274 │ │ │ max_columns=max_columns, │ │ │ │ /home/runner/.cache/pypoetry/virtualenvs/letsql-uJiiVu6z-py3.10/lib/python3.10/site-packages/ibi │ │ s/expr/types/pretty.py:342 in _to_rich_table │ │ │ │ 339 │ │ if orig_ncols > len(computed_cols): │ │ 340 │ │ │ table = table.select(*computed_cols) │ │ 341 │ │ │ ❱ 342 │ result = table.limit(max_rows + 1).to_pyarrow() │ │ 343 │ # Now format the columns in order, stopping if the console width would │ │ 344 │ # be exceeded. │ │ 345 │ col_info = [] │ │ │ │ /home/runner/.cache/pypoetry/virtualenvs/letsql-uJiiVu6z-py3.10/lib/python3.10/site-packages/too │ │ lz/functoolz.py:304 in __call__ │ │ │ │ 301 │ │ │ 302 │ def __call__(self, *args, **kwargs): │ │ 303 │ │ try: │ │ ❱ 304 │ │ │ return self._partial(*args, **kwargs) │ │ 305 │ │ except TypeError as exc: │ │ 306 │ │ │ if self._should_curry(args, kwargs, exc): │ │ 307 │ │ │ │ return self.bind(*args, **kwargs) │ │ │ │ /home/runner/work/letsql/letsql/python/letsql/backends/let/hotfix.py:196 in letsql_invoke │ │ │ │ 193 │ │ if dt not in con._sources.sources: │ │ 194 │ │ │ con.register(dt.to_expr(), dt.name) │ │ 195 │ method = getattr(con, f"{_methodname}") │ │ ❱ 196 │ return method(self, *args, **kwargs) │ │ 197 │ │ 198 │ │ 199 for typ, methodnames in ( │ │ │ │ /home/runner/work/letsql/letsql/python/letsql/backends/let/__init__.py:233 in to_pyarrow │ │ │ │ 230 │ │ │ 231 │ def to_pyarrow(self, expr: ir.Expr, **kwargs: Any) -> pa.Table: │ │ 232 │ │ backend, expr = self._get_backend_and_expr(expr) │ │ ❱ 233 │ │ return backend.to_pyarrow(expr.unbind(), **kwargs) │ │ 234 │ │ │ 235 │ def to_pyarrow_batches( │ │ 236 │ │ self, │ │ │ │ /home/runner/work/letsql/letsql/python/letsql/backends/datafusion/__init__.py:727 in to_pyarrow │ │ │ │ 724 │ │ │ 725 │ def to_pyarrow(self, expr: ir.Expr, **kwargs: Any) -> pa.Table: │ │ 726 │ │ batch_reader = self._to_pyarrow_batches(expr, **kwargs) │ │ ❱ 727 │ │ arrow_table = batch_reader.read_all() │ │ 728 │ │ return expr.__pyarrow_result__(arrow_table) │ │ 729 │ │ │ 730 │ def execute(self, expr: ir.Expr, **kwargs: Any): │ │ │ │ in pyarrow.lib.RecordBatchReader.read_all:757 │ │ │ │ in pyarrow.lib.check_status:88 │ │ │ │ /home/runner/work/letsql/letsql/python/letsql/backends/datafusion/__init__.py:717 in make_gen │ │ │ │ 714 │ │ │ │ │ .to_struct_array() │ │ 715 │ │ │ │ │ .cast(struct_schema) │ │ 716 │ │ │ │ ) │ │ ❱ 717 │ │ │ │ for batch in frame.execute_stream() │ │ 718 │ │ │ ) │ │ 719 │ │ │ │ 720 │ │ return pa.ipc.RecordBatchReader.from_batches( │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ Exception: This feature is not implemented: Physical plan does not support logical expression InSubquery(InSubquery { expr: Column(Column { relation: Some(Bare { table: "t0" }), name: "a" }), subquery: <subquery>, negated: false })
Exception: This feature is not implemented: Physical plan does not support logical expression InSubquery(InSubquery { expr: Column(Column { relation: Some(Bare { table: "t0" }), name: "a" }), subquery: <subquery>, negated: false })Exception: This feature is not implemented: Physical plan does not support logical expression InSubquery(InSubquery { expr: Column(Column { relation: Some(Bare { table: "t0" }), name: "a" }), subquery: <subquery>, negated: false })NULL behavior
┏━━━━━━━━━━━━━┓ ┃ InValues(x) ┃ ┡━━━━━━━━━━━━━┩ │ boolean │ ├─────────────┤ │ True │ │ NULL │ └─────────────┘
┏━━━━━━━━━━━━━┓ ┃ InValues(x) ┃ ┡━━━━━━━━━━━━━┩ │ boolean │ ├─────────────┤ │ True │ │ NULL │ │ False │ └─────────────┘
isnull()
Return whether this expression is NULL.
name(name)
Rename an expression to name.
| Name | Type | Description | Default |
|---|---|---|---|
name |
The new name of the expression | required |
| Type | Description |
|---|---|
| Value | self with name name |
notnull()
Return whether this expression is not NULL.
nullif(null_if_expr)
Set values to null if they equal the values null_if_expr.
Commonly used to avoid divide-by-zero problems by replacing zero with NULL in the divisor.
Equivalent to (self == null_if_expr).ifelse(ibis.null(), self).
| Name | Type | Description | Default |
|---|---|---|---|
null_if_expr |
Value | Expression indicating what values should be NULL | required |
| Type | Description |
|---|---|
| Value | Value expression |
try_cast(target_type)
Try cast expression to indicated data type.
If the cast fails for a row, the value is returned as null or NaN depending on target_type and backend behavior.
| Name | Type | Description | Default |
|---|---|---|---|
target_type |
Any | Type to try cast to. Anything accepted by ibis.dtype() |
required |
| Type | Description |
|---|---|
| Value | Casted expression |
>>> import ibis
>>> from ibis import _
>>> ibis.options.interactive = True
>>> t = ibis.memtable({"numbers": [1, 2, 3, 4], "strings": ["1.0", "2", "hello", "world"]})
>>> t┏━━━━━━━━━┳━━━━━━━━━┓ ┃ numbers ┃ strings ┃ ┡━━━━━━━━━╇━━━━━━━━━┩ │ int64 │ string │ ├─────────┼─────────┤ │ 1 │ 1.0 │ │ 2 │ 2 │ │ 3 │ hello │ │ 4 │ world │ └─────────┴─────────┘
>>> t = t.mutate(numbers_to_strings=_.numbers.try_cast("string"))
>>> t = t.mutate(strings_to_numbers=_.strings.try_cast("int"))
>>> t╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ /home/runner/.cache/pypoetry/virtualenvs/letsql-uJiiVu6z-py3.10/lib/python3.10/site-packages/ibi │ │ s/expr/types/core.py:99 in __rich_console__ │ │ │ │ 96 │ │ │ │ 97 │ │ try: │ │ 98 │ │ │ if opts.interactive: │ │ ❱ 99 │ │ │ │ rich_object = to_rich(self, console_width=console_width) │ │ 100 │ │ │ else: │ │ 101 │ │ │ │ rich_object = Text(self._noninteractive_repr()) │ │ 102 │ │ except Exception as e: │ │ │ │ /home/runner/.cache/pypoetry/virtualenvs/letsql-uJiiVu6z-py3.10/lib/python3.10/site-packages/ibi │ │ s/expr/types/pretty.py:271 in to_rich │ │ │ │ 268 │ │ │ expr, max_length=max_length, max_string=max_string, max_depth=max_depth │ │ 269 │ │ ) │ │ 270 │ else: │ │ ❱ 271 │ │ return _to_rich_table( │ │ 272 │ │ │ expr, │ │ 273 │ │ │ max_rows=max_rows, │ │ 274 │ │ │ max_columns=max_columns, │ │ │ │ /home/runner/.cache/pypoetry/virtualenvs/letsql-uJiiVu6z-py3.10/lib/python3.10/site-packages/ibi │ │ s/expr/types/pretty.py:342 in _to_rich_table │ │ │ │ 339 │ │ if orig_ncols > len(computed_cols): │ │ 340 │ │ │ table = table.select(*computed_cols) │ │ 341 │ │ │ ❱ 342 │ result = table.limit(max_rows + 1).to_pyarrow() │ │ 343 │ # Now format the columns in order, stopping if the console width would │ │ 344 │ # be exceeded. │ │ 345 │ col_info = [] │ │ │ │ /home/runner/.cache/pypoetry/virtualenvs/letsql-uJiiVu6z-py3.10/lib/python3.10/site-packages/too │ │ lz/functoolz.py:304 in __call__ │ │ │ │ 301 │ │ │ 302 │ def __call__(self, *args, **kwargs): │ │ 303 │ │ try: │ │ ❱ 304 │ │ │ return self._partial(*args, **kwargs) │ │ 305 │ │ except TypeError as exc: │ │ 306 │ │ │ if self._should_curry(args, kwargs, exc): │ │ 307 │ │ │ │ return self.bind(*args, **kwargs) │ │ │ │ /home/runner/work/letsql/letsql/python/letsql/backends/let/hotfix.py:196 in letsql_invoke │ │ │ │ 193 │ │ if dt not in con._sources.sources: │ │ 194 │ │ │ con.register(dt.to_expr(), dt.name) │ │ 195 │ method = getattr(con, f"{_methodname}") │ │ ❱ 196 │ return method(self, *args, **kwargs) │ │ 197 │ │ 198 │ │ 199 for typ, methodnames in ( │ │ │ │ /home/runner/work/letsql/letsql/python/letsql/backends/let/__init__.py:233 in to_pyarrow │ │ │ │ 230 │ │ │ 231 │ def to_pyarrow(self, expr: ir.Expr, **kwargs: Any) -> pa.Table: │ │ 232 │ │ backend, expr = self._get_backend_and_expr(expr) │ │ ❱ 233 │ │ return backend.to_pyarrow(expr.unbind(), **kwargs) │ │ 234 │ │ │ 235 │ def to_pyarrow_batches( │ │ 236 │ │ self, │ │ │ │ /home/runner/work/letsql/letsql/python/letsql/backends/datafusion/__init__.py:727 in to_pyarrow │ │ │ │ 724 │ │ │ 725 │ def to_pyarrow(self, expr: ir.Expr, **kwargs: Any) -> pa.Table: │ │ 726 │ │ batch_reader = self._to_pyarrow_batches(expr, **kwargs) │ │ ❱ 727 │ │ arrow_table = batch_reader.read_all() │ │ 728 │ │ return expr.__pyarrow_result__(arrow_table) │ │ 729 │ │ │ 730 │ def execute(self, expr: ir.Expr, **kwargs: Any): │ │ │ │ in pyarrow.lib.RecordBatchReader.read_all:757 │ │ │ │ in pyarrow.lib.check_status:88 │ │ │ │ /home/runner/work/letsql/letsql/python/letsql/backends/datafusion/__init__.py:705 in make_gen │ │ │ │ 702 │ │ struct_schema = schema.as_struct().to_pyarrow() │ │ 703 │ │ │ │ 704 │ │ def make_gen(): │ │ ❱ 705 │ │ │ yield from ( │ │ 706 │ │ │ │ # convert the renamed + cast columns into a record batch │ │ 707 │ │ │ │ pa.RecordBatch.from_struct_array( │ │ 708 │ │ │ │ │ # rename columns to match schema because datafusion lowercases thing │ │ │ │ /home/runner/work/letsql/letsql/python/letsql/backends/datafusion/__init__.py:705 in <genexpr> │ │ │ │ 702 │ │ struct_schema = schema.as_struct().to_pyarrow() │ │ 703 │ │ │ │ 704 │ │ def make_gen(): │ │ ❱ 705 │ │ │ yield from ( │ │ 706 │ │ │ │ # convert the renamed + cast columns into a record batch │ │ 707 │ │ │ │ pa.RecordBatch.from_struct_array( │ │ 708 │ │ │ │ │ # rename columns to match schema because datafusion lowercases thing │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ Exception: Arrow error: Cast error: Cannot cast string '1.0' to value of Int64 type
Exception: Arrow error: Cast error: Cannot cast string '1.0' to value of Int64 typeException: Arrow error: Cast error: Cannot cast string '1.0' to value of Int64 typeScalar(self, arg)
| Name | Description |
|---|---|
| as_table | Promote the scalar expression to a table. |
as_table()
Promote the scalar expression to a table.
| Type | Description |
|---|---|
| Table | A table expression |
Promote an aggregation to a table
>>> import ibis
>>> import ibis.expr.types as ir
>>> t = ibis.table(dict(a="str"), name="t")
>>> expr = t.a.length().sum().name("len").as_table()
>>> isinstance(expr, ir.Table)TruePromote a literal value to a table
Column(self, arg)
| Name | Description |
|---|---|
| approx_median | Return an approximate of the median of self. |
| approx_nunique | Return the approximate number of distinct elements in self. |
| arbitrary | Select an arbitrary value in a column. |
| count | Compute the number of rows in an expression. |
| first | Return the first value of a column. |
| lag | Return the row located at offset rows before the current row. |
| last | Return the last value of a column. |
| lead | Return the row located at offset rows after the current row. |
| max | Return the maximum of a column. |
| median | Return the median of the column. |
| min | Return the minimum of a column. |
| nth | Return the nth value (0-indexed) over a window. |
| nunique | Compute the number of distinct rows in an expression. |
approx_median(where=None)
Return an approximate of the median of self.
Whether the result is an approximation depends on the backend.
| Name | Type | Description | Default |
|---|---|---|---|
where |
ir.BooleanValue | None | Filter in values when where is True |
None |
| Type | Description |
|---|---|
| Scalar | An approximation of the median of self |
approx_nunique(where=None)
Return the approximate number of distinct elements in self.
Whether the result is an approximation depends on the backend.
| Name | Type | Description | Default |
|---|---|---|---|
where |
ir.BooleanValue | None | Filter in values when where is True |
None |
| Type | Description |
|---|---|
| Scalar | An approximate count of the distinct elements of self |
arbitrary(where=None, how=None)
Select an arbitrary value in a column.
Returns an arbitrary (nondeterministic, backend-specific) value from the column. The value will be non-NULL, except if the column is empty or all values are NULL.
| Name | Type | Description | Default |
|---|---|---|---|
where |
ir.BooleanValue | None | A filter expression | None |
how |
Any | DEPRECATED | None |
| Type | Description |
|---|---|
| Scalar | An expression |
count(where=None)
Compute the number of rows in an expression.
| Name | Type | Description | Default |
|---|---|---|---|
where |
ir.BooleanValue | None | Filter expression | None |
| Type | Description |
|---|---|
| IntegerScalar | Number of elements in an expression |
first(where=None)
Return the first value of a column.
lag(offset=None, default=None)
Return the row located at offset rows before the current row.
| Name | Type | Description | Default |
|---|---|---|---|
offset |
int | ir.IntegerValue | None | Index of row to select | None |
default |
Value | None | Value used if no row exists at offset |
None |
last(where=None)
Return the last value of a column.
lead(offset=None, default=None)
Return the row located at offset rows after the current row.
| Name | Type | Description | Default |
|---|---|---|---|
offset |
int | ir.IntegerValue | None | Index of row to select | None |
default |
Value | None | Value used if no row exists at offset |
None |
max(where=None)
Return the maximum of a column.
| Name | Type | Description | Default |
|---|---|---|---|
where |
ir.BooleanValue | None | Filter in values when where is True |
None |
| Type | Description |
|---|---|
| Scalar | The maximum value in self |
median(where=None)
Return the median of the column.
| Name | Type | Description | Default |
|---|---|---|---|
where |
ir.BooleanValue | None | Optional boolean expression. If given, only the values where where evaluates to true will be considered for the median. |
None |
| Type | Description |
|---|---|
| Scalar | Median of the column |
Compute the median of bill_depth_mm
>>> t.group_by(t.species).agg(median_bill_depth=t.bill_depth_mm.median()).order_by(
... ibis.desc("median_bill_depth")
... )┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┓ ┃ species ┃ median_bill_depth ┃ ┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━┩ │ string │ float64 │ ├───────────┼───────────────────┤ │ Chinstrap │ 18.45 │ │ Adelie │ 18.40 │ │ Gentoo │ 15.00 │ └───────────┴───────────────────┘
In addition to numeric types, any orderable non-numeric types such as strings and dates work with median.
>>> t.group_by(t.island).agg(median_species=t.species.median()).order_by(
... ibis.desc("median_species")
... )╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ /home/runner/.cache/pypoetry/virtualenvs/letsql-uJiiVu6z-py3.10/lib/python3.10/site-packages/ibi │ │ s/expr/types/core.py:99 in __rich_console__ │ │ │ │ 96 │ │ │ │ 97 │ │ try: │ │ 98 │ │ │ if opts.interactive: │ │ ❱ 99 │ │ │ │ rich_object = to_rich(self, console_width=console_width) │ │ 100 │ │ │ else: │ │ 101 │ │ │ │ rich_object = Text(self._noninteractive_repr()) │ │ 102 │ │ except Exception as e: │ │ │ │ /home/runner/.cache/pypoetry/virtualenvs/letsql-uJiiVu6z-py3.10/lib/python3.10/site-packages/ibi │ │ s/expr/types/pretty.py:271 in to_rich │ │ │ │ 268 │ │ │ expr, max_length=max_length, max_string=max_string, max_depth=max_depth │ │ 269 │ │ ) │ │ 270 │ else: │ │ ❱ 271 │ │ return _to_rich_table( │ │ 272 │ │ │ expr, │ │ 273 │ │ │ max_rows=max_rows, │ │ 274 │ │ │ max_columns=max_columns, │ │ │ │ /home/runner/.cache/pypoetry/virtualenvs/letsql-uJiiVu6z-py3.10/lib/python3.10/site-packages/ibi │ │ s/expr/types/pretty.py:342 in _to_rich_table │ │ │ │ 339 │ │ if orig_ncols > len(computed_cols): │ │ 340 │ │ │ table = table.select(*computed_cols) │ │ 341 │ │ │ ❱ 342 │ result = table.limit(max_rows + 1).to_pyarrow() │ │ 343 │ # Now format the columns in order, stopping if the console width would │ │ 344 │ # be exceeded. │ │ 345 │ col_info = [] │ │ │ │ /home/runner/.cache/pypoetry/virtualenvs/letsql-uJiiVu6z-py3.10/lib/python3.10/site-packages/too │ │ lz/functoolz.py:304 in __call__ │ │ │ │ 301 │ │ │ 302 │ def __call__(self, *args, **kwargs): │ │ 303 │ │ try: │ │ ❱ 304 │ │ │ return self._partial(*args, **kwargs) │ │ 305 │ │ except TypeError as exc: │ │ 306 │ │ │ if self._should_curry(args, kwargs, exc): │ │ 307 │ │ │ │ return self.bind(*args, **kwargs) │ │ │ │ /home/runner/work/letsql/letsql/python/letsql/backends/let/hotfix.py:196 in letsql_invoke │ │ │ │ 193 │ │ if dt not in con._sources.sources: │ │ 194 │ │ │ con.register(dt.to_expr(), dt.name) │ │ 195 │ method = getattr(con, f"{_methodname}") │ │ ❱ 196 │ return method(self, *args, **kwargs) │ │ 197 │ │ 198 │ │ 199 for typ, methodnames in ( │ │ │ │ /home/runner/work/letsql/letsql/python/letsql/backends/let/__init__.py:233 in to_pyarrow │ │ │ │ 230 │ │ │ 231 │ def to_pyarrow(self, expr: ir.Expr, **kwargs: Any) -> pa.Table: │ │ 232 │ │ backend, expr = self._get_backend_and_expr(expr) │ │ ❱ 233 │ │ return backend.to_pyarrow(expr.unbind(), **kwargs) │ │ 234 │ │ │ 235 │ def to_pyarrow_batches( │ │ 236 │ │ self, │ │ │ │ /home/runner/work/letsql/letsql/python/letsql/backends/datafusion/__init__.py:727 in to_pyarrow │ │ │ │ 724 │ │ │ 725 │ def to_pyarrow(self, expr: ir.Expr, **kwargs: Any) -> pa.Table: │ │ 726 │ │ batch_reader = self._to_pyarrow_batches(expr, **kwargs) │ │ ❱ 727 │ │ arrow_table = batch_reader.read_all() │ │ 728 │ │ return expr.__pyarrow_result__(arrow_table) │ │ 729 │ │ │ 730 │ def execute(self, expr: ir.Expr, **kwargs: Any): │ │ │ │ in pyarrow.lib.RecordBatchReader.read_all:757 │ │ │ │ in pyarrow.lib.check_status:88 │ │ │ │ /home/runner/work/letsql/letsql/python/letsql/backends/datafusion/__init__.py:705 in make_gen │ │ │ │ 702 │ │ struct_schema = schema.as_struct().to_pyarrow() │ │ 703 │ │ │ │ 704 │ │ def make_gen(): │ │ ❱ 705 │ │ │ yield from ( │ │ 706 │ │ │ │ # convert the renamed + cast columns into a record batch │ │ 707 │ │ │ │ pa.RecordBatch.from_struct_array( │ │ 708 │ │ │ │ │ # rename columns to match schema because datafusion lowercases thing │ │ │ │ /home/runner/work/letsql/letsql/python/letsql/backends/datafusion/__init__.py:705 in <genexpr> │ │ │ │ 702 │ │ struct_schema = schema.as_struct().to_pyarrow() │ │ 703 │ │ │ │ 704 │ │ def make_gen(): │ │ ❱ 705 │ │ │ yield from ( │ │ 706 │ │ │ │ # convert the renamed + cast columns into a record batch │ │ 707 │ │ │ │ pa.RecordBatch.from_struct_array( │ │ 708 │ │ │ │ │ # rename columns to match schema because datafusion lowercases thing │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ Exception: External error: External error: This feature is not implemented: MedianAccumulator not supported for median(t0.species) with Utf8
Exception: External error: External error: This feature is not implemented: MedianAccumulator not supported for median(t0.species) with Utf8Exception: External error: External error: This feature is not implemented: MedianAccumulator not supported for median(t0.species) with Utf8min(where=None)
Return the minimum of a column.
| Name | Type | Description | Default |
|---|---|---|---|
where |
ir.BooleanValue | None | Filter in values when where is True |
None |
| Type | Description |
|---|---|
| Scalar | The minimum value in self |
nth(n)
Return the nth value (0-indexed) over a window.
.nth(0) is equivalent to .first(). Negative will result in NULL. If the value of n is greater than the number of rows in the window, NULL will be returned.
| Name | Type | Description | Default |
|---|---|---|---|
n |
int | ir.IntegerValue | Desired rank value | required |
| Type | Description |
|---|---|
| Column | The nth value over a window |
nunique(where=None)
Compute the number of distinct rows in an expression.
| Name | Type | Description | Default |
|---|---|---|---|
where |
ir.BooleanValue | None | Filter expression | None |
| Type | Description |
|---|---|
| IntegerScalar | Number of distinct elements in an expression |