iris-embedded-python-wrapper

This is a module that wraps embedded python in the IRIS Dataplateform. It provides a simple interface to run python code in IRIS.

More details can be found in the IRIS documentation

Pre-requisites

To make use of this module, you need to have the IRIS Dataplatform installed on your machine (more details can be found here).

Then you must configure the service callin to allow the python code to be executed and set the environment variables.

Configuration of the service callin

In the Management Portal, go to System Administration > Security > Services, select %Service_CallIn, and check the Service Enabled box.

More details can be found in the IRIS documentation

Environment Variables

Set the following environment variables :

IRISINSTALLDIR: The path to the IRIS installation directory
LD_LIBRARY_PATH: The Linux loader path for IRIS shared libraries
DYLD_LIBRARY_PATH: The macOS loader path for IRIS shared libraries
IRISUSERNAME: The username to connect to IRIS
IRISPASSWORD: The password to connect to IRIS
IRISNAMESPACE: The namespace to connect to IRIS

Embedded-local execution from regular python3 on Unix needs the loader path configured before Python starts. iris.connect(path=...) can configure Python import paths at runtime, but it cannot repair Unix dynamic loader resolution after startup. When embedded-local loading is explicitly requested, the wrapper emits a RuntimeWarning on Unix if the loader-path variable does not already include the IRIS bin directory.

For Linux and MacOS

For Linux and MacOS, you can set the environment variables as follows:

export IRISINSTALLDIR=/opt/iris
export LD_LIBRARY_PATH=$IRISINSTALLDIR/bin:$LD_LIBRARY_PATH
# for MacOS
export DYLD_LIBRARY_PATH=$IRISINSTALLDIR/bin:$DYLD_LIBRARY_PATH
# for IRIS username
export IRISUSERNAME=SuperUser
export IRISPASSWORD=SYS
export IRISNAMESPACE=USER

Warning: when embedded-local and the Native API wheel run in the same Python process, loader-path ordering matters. pythonint needs shared libraries from the IRIS bin directory, while the Native API wheel needs its bundled ELS SDK libraries first. See Native API wheel and IRIS bin loader-path conflict.

For windows

For windows, you can set the environment variables as follows:

set IRISINSTALLDIR=C:\path\to\iris
set LD_LIBRARY_PATH=%IRISINSTALLDIR%\bin;%LD_LIBRARY_PATH%

For Python 3.8 and newer, the wrapper automatically registers the IRIS bin directory with os.add_dll_directory() when IRISINSTALLDIR is set. Update PATH only when using older Python versions or external tools that need IRIS DLLs:

set PATH=%IRISINSTALLDIR%\bin;%PATH%

Set the IRIS username, password, and namespace

set IRISUSERNAME=SuperUser
set IRISPASSWORD=SYS
set IRISNAMESPACE=USER

For PowerShell

For PowerShell, you can set the environment variables as follows:

$env:IRISINSTALLDIR="C:\path\to\iris"
$env:IRISUSERNAME="SuperUser"
$env:IRISPASSWORD="SYS"
$env:IRISNAMESPACE="USER"

Installation

pip install iris-embedded-python-wrapper

Usage

You can use this module in three ways:

Run python code in IRIS
Bind a virtual environment to embedded python in IRIS
Unbind a virtual environment from embedded python in IRIS

Run python code in IRIS

Now you can use the module to run python code in IRIS. Here is an example:

import iris
iris.system.Version.GetVersion()

Output:

'IRIS for UNIX (Apple Mac OS X for x86-64) 2024.3 (Build 217U) Thu Nov 14 2024 17:29:23 EST'

Unified runtime context

The wrapper now uses a unified runtime API through iris.runtime.

Runtime model

iris.runtime.mode: selected policy (auto, embedded, native)
iris.runtime.state: detected runtime (embedded-kernel, embedded-local, native-remote, unavailable)
iris.runtime.embedded_available: whether embedded backend can be used
iris.runtime.iris: currently bound native object API handle (optional)
iris.runtime.dbapi: optional explicitly bound DB-API connection

Runtime control API

iris.runtime.get()
iris.runtime.configure(mode="auto", install_dir=None, iris=None, dbapi=None, native_connection=None)
iris.runtime.reset()

mode is optional in runtime.configure(...).

If iris, native_connection, or dbapi is provided, runtime infers native mode.
If no connection handle is provided, runtime stays in auto/embedded detection flow.

runtime.configure(...) also accepts an IRISConnection and auto-converts it to an IRIS handle via createIRIS(...) for iris.cls(...) routing.

Examples

Force native object API routing:

import iris

conn = iris.createConnection("localhost", 1972, "USER", "_SYSTEM", "SYS")
db = iris.createIRIS(conn)
iris.runtime.configure(mode="native", iris=db, native_connection=conn)

obj = iris.cls("Ens.StringRequest")._New()

Native routing with inferred mode and auto-conversion from IRISConnection:

import iris

conn = iris.createConnection("localhost", 1972, "USER", "_SYSTEM", "SYS")
iris.runtime.configure(native_connection=conn)

obj = iris.cls("Ens.StringRequest")._New()

Force embedded routing:

import iris

iris.runtime.configure(mode="embedded")
obj = iris.cls("Ens.StringRequest")._New()

Enable embedded routing with an explicit IRIS installation directory:

import iris

iris.connect(path="/opt/iris")
obj = iris.cls("Ens.StringRequest")._New()

This is useful when IRISINSTALLDIR is not set. On Linux and macOS, the native library path still needs to be configured before Python starts as shown in the environment setup section. The path must point to an IRIS installation directory with bin and lib/python subdirectories; invalid paths fail before the wrapper mutates Python import paths. For explicit path=..., the wrapper also removes stale pythonint modules for the import attempt and verifies that the loaded pythonint.__file__ is under that installation's bin or lib/python directory.

iris.connect(path=...) returns the runtime context. If the loaded embedded backend does not expose a callable connect, the wrapper emits a RuntimeWarning; use iris.dbapi.connect(path=...) when you want a DB-API connection in one call.

Reset to automatic detection:

import iris

iris.runtime.reset()

DB-API (`iris.dbapi`)

The wrapper exposes a DB-API facade at iris.dbapi.

Supported subset

iris.dbapi.connect(...)
Connection: cursor(), close(), commit(), rollback()
Cursor: execute(), fetchone(), fetchmany(), fetchall(), iteration, close()
PEP 249 metadata: apilevel, threadsafety, paramstyle
PEP 249 exceptions: Error, InterfaceError, OperationalError, and related subclasses

Value normalization

For the embedded %SQL.Statement backend, the wrapper normalizes IRIS SQL/ObjectScript string boundary values to Python values so embedded and remote DB-API behave the same way:

SQL NULL is returned as Python None
SQL empty string is returned as Python ""
Python None passed as a parameter remains SQL NULL
Python "" passed as a parameter is written as an SQL empty string, not SQL NULL

This normalization is limited to the embedded DB-API path. Native/remote DB-API values are returned by the official driver.

For the native object proxy path (iris.cls(...) with iris.runtime configured for native mode), the wrapper also normalizes declared scalar string properties:

%String / %RawString scalar properties that come back as None from the native proxy are returned as Python ""
non-string properties are left unchanged
collection-valued properties are left unchanged
arbitrary method return values are left unchanged

Connect modes

iris.dbapi.connect() accepts mode="auto" | "embedded" | "native".

mode="embedded": forces embedded SQL backend via %SQL.Statement
mode="native": forces native DB-API backend via the official module iris.dbapi
mode="auto":
- if explicit remote arguments are provided (hostname, port, namespace, etc.), uses native
- if iris.runtime.dbapi is already bound, reuses that DB-API connection
- otherwise uses embedded (%SQL.Statement) only when runtime policy is not native
- if iris.runtime is configured for native mode without a bound DB-API connection, raises an error instead of silently falling back to embedded
- raises an error if embedded runtime is not available

Native resolution uses the official module path iris.dbapi (not intersystems_iris.dbapi).

mode is optional for DB-API.

With explicit remote arguments (hostname, port, namespace, username, password, etc.), DB-API infers native.
With path=..., DB-API configures embedded-local runtime and returns an embedded DB-API connection. mode must be auto or embedded.
With iris.runtime.configure(dbapi=conn), DB-API auto mode reuses the bound native connection.
Without remote arguments or a bound runtime DB-API connection, DB-API auto mode uses embedded unless iris.runtime is explicitly in native mode.

iris.connect(path=...) and iris.dbapi.connect(path=...) share the same embedded runtime configuration behavior, but return different things:

iris.connect(path=...) returns the RuntimeContext
iris.dbapi.connect(path=...) returns a DB-API connection

iris.dbapi.connect(path=...) accepts embedded DB-API options such as namespace=... and isolation_level=.... It rejects native mode and native connection arguments such as hostname, port, username, and password. The path is validated with the same rules as iris.connect(path=...).

Examples

Embedded mode:

import iris

conn = iris.dbapi.connect(mode="embedded")
cur = conn.cursor()
cur.execute("SELECT Name FROM Sample.Person")
rows = cur.fetchall()
cur.close()
conn.close()

Embedded-local mode with an explicit IRIS installation directory:

import iris

conn = iris.dbapi.connect(path="/opt/iris", namespace="USER")
cur = conn.cursor()
cur.execute("SELECT 1")
print(cur.fetchone())

Native mode:

import iris

conn = iris.dbapi.connect(
        mode="native",
        hostname="localhost",
        port=1972,
        namespace="USER",
        username="_SYSTEM",
        password="SYS",
)
cur = conn.cursor()
cur.execute("SELECT 1")
print(cur.fetchone())

Auto mode with explicit remote arguments (routes to native):

import iris

conn = iris.dbapi.connect(
        hostname="localhost",
        port=1972,
        namespace="USER",
        username="_SYSTEM",
        password="SYS",
)

Auto mode with a runtime-bound native DB-API connection:

import iris

conn = iris.dbapi.connect(
        mode="native",
        hostname="localhost",
        port=1972,
        namespace="USER",
        username="_SYSTEM",
        password="SYS",
)
iris.runtime.configure(dbapi=conn)

same_conn = iris.dbapi.connect(mode="auto")
assert same_conn is conn

Runtime independence

iris.dbapi.connect() is independent from iris.runtime by default.

Calling iris.dbapi.connect(...) does not auto-bind a connection into iris.runtime.dbapi. If you need runtime-managed DB-API binding, bind it explicitly with iris.runtime.configure(dbapi=conn). Once bound, iris.dbapi.connect(mode="auto") reuses that connection instead of creating a new one.

Bind a virtual environment to embedded python in IRIS

You can also bind or unbind an virtual environment to embedded python in IRIS. Here is an example:

bind_iris

Output:

(.venv) demo ‹master*›$ bind_iris
INFO:iris_utils._find_libpython:Created backup at /opt/intersystems/iris/iris.cpf.fa76423a7b924eb085911690c8266129
INFO:iris_utils._find_libpython:Created merge file at /opt/intersystems/iris/iris.cpf.python_merge
up  IRIS              2024.3.0.217.0    1972   /opt/intersystems/iris

Username: SuperUser
Password: ***
IRIS Merge of /opt/intersystems/iris/iris.cpf.python_merge into /opt/intersystems/iris/iris.cpf
IRIS Merge completed successfully
INFO:iris_utils._find_libpython:PythonRuntimeLibrary path set to /usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/Python
INFO:iris_utils._find_libpython:PythonPath set to /demo/.venv/lib/python3.11/site-packages
INFO:iris_utils._find_libpython:PythonRuntimeLibraryVersion set to 3.11

You may have to put your admin credentials to bind the virtual environment to the embedded python in IRIS.

In windows, you must restart the IRIS.

Unbind a virtual environment from embedded python in IRIS

unbind_iris

Output:

(.venv) demo ‹master*›$ unbind_iris
INFO:iris_utils._find_libpython:Created merge file at /opt/intersystems/iris/iris.cpf.python_merge
up  IRIS              2024.3.0.217.0    1972   /opt/intersystems/iris

Username: SuperUser
Password: ***
IRIS Merge of /opt/intersystems/iris/iris.cpf.python_merge into /opt/intersystems/iris/iris.cpf
IRIS Merge completed successfully
INFO:iris_utils._find_libpython:PythonRuntimeLibrary path set to /usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/Python
INFO:iris_utils._find_libpython:PythonPath set to /Other/.venv/lib/python3.11/site-packages
INFO:iris_utils._find_libpython:PythonRuntimeLibraryVersion set to 3.11

Known Issues

Native API wheel and IRIS `bin` loader-path conflict

If intersystems-irispython is installed in the same environment, be careful with LD_LIBRARY_PATH and DYLD_LIBRARY_PATH. The Native API wheel ships its own ELS SDK libraries, and an IRIS installation also contains libraries with the same names, such as libelsdkcore.dylib on macOS. If the IRIS bin directory appears first in the loader path, the Native API extension can bind to the IRIS installation library instead of the wheel's bundled, ABI-compatible library. This can fail while importing iris.irissdk with errors such as Symbol not found or undefined symbol.

Embedded-local mode still needs the IRIS bin directory in the loader path because pythonint depends on shared libraries from the IRIS installation. The conflict is ordering: pythonint needs IRIS libraries visible, but the Native API wheel must resolve its ELS SDK libraries from the wheel before the dynamic loader searches the IRIS bin directory.

When embedded-local and the Native API must run in the same Python process, put the wheel's bundled native library directory before the IRIS bin directory, and do it before Python starts:

export IRISINSTALLDIR=/opt/iris
export IRIS_WHEEL_LIBS=$(python - <<'PY'
from importlib import metadata
from pathlib import Path

iris_dir = Path(metadata.distribution("intersystems-irispython").locate_file("iris"))
for name in (".dylibs", ".libs"):
    candidate = iris_dir / name
    if candidate.is_dir():
        print(candidate)
        break
PY
)
export DYLD_LIBRARY_PATH=$IRIS_WHEEL_LIBS:$IRISINSTALLDIR/bin:$DYLD_LIBRARY_PATH
# Linux: use LD_LIBRARY_PATH with the same ordering.

Changing these variables from inside Python cannot repair the current process. If a process only uses the Native API wheel and does not load embedded Python, do not point the loader path at an incompatible IRIS bin directory.

Troubleshooting

You may encounter the following error, here is how to fix them.

No module named 'pythonint'

This usually means the wrapper cannot find the IRIS embedded Python extension. Check that IRISINSTALLDIR or path=... points to the IRIS installation directory and that it contains both bin and lib/python.

If the error mentions IRIS shared libraries, configure the platform loader path before Python starts:

export IRISINSTALLDIR=/opt/iris
export LD_LIBRARY_PATH=$IRISINSTALLDIR/bin:$LD_LIBRARY_PATH
# macOS
export DYLD_LIBRARY_PATH=$IRISINSTALLDIR/bin:$DYLD_LIBRARY_PATH

IRIS_ACCESSDENIED (-15)

This can occur when the service callin is not enabled. Make sure that the service callin is enabled.

IRIS_ATTACH (-21)

This can occur when the user is not the same as the iris owner. Make sure that the user is the same as the iris owner.

iris-embedded-python-wrapper

Pre-requisites

Configuration of the service callin

Environment Variables

For Linux and MacOS

For windows

For PowerShell

Installation

Usage

Run python code in IRIS

Unified runtime context

Runtime model

Runtime control API

Examples

DB-API (iris.dbapi)

Supported subset

Value normalization

Connect modes

Examples

Runtime independence

Bind a virtual environment to embedded python in IRIS

Unbind a virtual environment from embedded python in IRIS

Known Issues

Native API wheel and IRIS bin loader-path conflict

Troubleshooting

No module named 'pythonint'

IRIS_ACCESSDENIED (-15)

IRIS_ATTACH (-21)

DB-API (`iris.dbapi`)

Native API wheel and IRIS `bin` loader-path conflict