Vector Manager

This section documents the vector management classes that provide vector operations and indexing capabilities.

VectorManager

The VectorManager class provides comprehensive vector operations including similarity search, range search, and vector indexing capabilities.

class matrixone.client.VectorManager(client, executor=None)

Bases: _VectorManagerBase

Synchronous vector manager for MatrixOne vector operations.

Supports two usage modes: 1. Client mode: Each operation is an independent auto-commit transaction 2. Session mode: Operations execute within the session’s transaction context

__init__(client, executor=None)

Initialize VectorManager.

create_ivf(table_name: str | type, name: str, column: str, lists: int = 100, op_type: VectorOpType = None) → VectorManager

Create an IVFFLAT vector index.

create_hnsw(table_name: str | type, name: str, column: str, m: int = 16, ef_construction: int = 200, ef_search: int = 50, op_type: VectorOpType = None) → VectorManager

Create an HNSW vector index.

drop(table_name: str | type, name: str) → VectorManager

Drop a vector index.

enable_ivf(probe_limit: int = 1) → VectorManager

Enable IVF indexing.

disable_ivf() → VectorManager

Disable IVF indexing.

enable_hnsw() → VectorManager

Enable HNSW indexing.

disable_hnsw() → VectorManager

Disable HNSW indexing.

insert(table_name: str | type, data: Dict[str, Any]) → VectorManager

Insert vector data into table.

batch_insert(table_name: str | type, data_list: List[Dict[str, Any]]) → VectorManager

Batch insert vector data.

similarity_search(table_name: str | type, vector_column: str, query_vector: List[float], limit: int = 10, select_columns: List[str] | None = None, where_clause: str | None = None, distance_type: str = 'l2', log_mode: str | None = None) → List[Dict[str, Any]]

Perform vector similarity search.

range_search(table_name: str | type, vector_column: str, query_vector: List[float], max_distance: float, select_columns: List[str] | None = None, where_clause: str | None = None, log_mode: str | None = None) → List[Dict[str, Any]]

Perform vector range search.

search_with_rank(table_name: str | type, vector_column: str, query_vector: List[float], limit: int = 10, select_columns: List[str] = None, where_clause: str = None, distance_type: str = 'l2', rank_mode: str | 'IVFRankMode' = None, log_mode: str = None) → List[Dict[str, Any]]

Perform vector similarity search with IVF LIMIT BY RANK support.

This method executes vector search using MatrixOne’s IVF index with fine-grained control over ranking strategy via the LIMIT BY RANK WITH OPTION clause.

Parameters:

• table_name – Name of the table (string) or SQLAlchemy Model class.

• vector_column – Name of the vector column to search.

• query_vector – Query vector as list of floats.

• limit – Maximum number of results to return. Defaults to 10.

• select_columns – List of columns to select. If None, selects all.

• where_clause – Optional WHERE clause for filtering before search.

• distance_type – Distance metric - “l2”, “cosine”, or “inner_product”. Defaults to “l2”.

• rank_mode – IVF ranking mode - “pre”, “post”, or “force”. - “pre”: Fast approximate search (default) - “post”: Slower but more accurate search - “force”: Force index usage with strict ranking If None, defaults to “post”.

• log_mode – Internal logging mode parameter.

Returns:

List of dictionaries containing search results with distance column.

Raises:

Exception – If search fails or invalid parameters provided.

Example

>>> # Complete runnable example
>>> from matrixone import Client, IVFRankMode
>>> client = Client()
>>> client.connect(database="test")
>>>
>>> # Create table
>>> client.execute('''
...     CREATE TABLE IF NOT EXISTS documents (
...         id INT PRIMARY KEY,
...         title VARCHAR(200),
...         embedding VECF32(4)
...     )
... ''')
>>>
>>> # Insert data
>>> client.execute("INSERT INTO documents VALUES (1, 'Doc 1', '[0.1,0.2,0.3,0.4]')")
>>> client.execute("INSERT INTO documents VALUES (2, 'Doc 2', '[0.2,0.3,0.4,0.5]')")
>>>
>>> # Create IVF index
>>> client.vector_ops.create_ivf('documents', 'idx_emb', 'embedding', lists=2)
>>>
>>> # Search with rank
>>> results = client.vector_ops.search_with_rank(
...     table_name="documents",
...     vector_column="embedding",
...     query_vector=[0.15, 0.25, 0.35, 0.45],
...     limit=2
... )
>>> len(results)
2
>>>
>>> # Cleanup
>>> client.execute("DROP TABLE documents")
>>> client.disconnect()

get_ivf_stats(table_name: str | type, column_name: str | None = None, database: str | None = None) → Dict[str, Any]

Get IVF index statistics.

Parameters:

• table_name – Name of the table (string) or SQLAlchemy Model class

• column_name – Optional vector column name (auto-inferred if not provided)

• database – Optional database name (uses current if not provided)

Returns:

Dictionary containing index statistics and distribution

Key Features: - Vector similarity search with configurable distance metrics (L2, cosine, inner product) - Vector range search for finding vectors within a distance threshold - Vector indexing support (IVF, HNSW) - Vector data insertion and management - Integration with MatrixOne’s vector functions

Supported Distance Metrics: - L2 (Euclidean) distance: Standard Euclidean distance - Cosine similarity: Cosine of the angle between vectors - Inner product: Dot product of vectors

Usage Examples:

# Initialize vector manager
vector_ops = client.vector_ops

# Similarity search with L2 distance
results = vector_ops.similarity_search(
    table_name_or_model="documents",
    vector_column="embedding",
    query_vector=[0.1, 0.2, 0.3, ...],  # 384-dimensional vector
    limit=10,
    distance_type="l2"
)

# Range search with cosine similarity
results = vector_ops.range_search(
    table_name_or_model="products",
    vector_column="features",
    query_vector=[0.5, 0.6, 0.7, ...],
    max_distance=0.8,
    distance_type="cosine"
)

VectorTableBuilder

class matrixone.sqlalchemy_ext.table_builder.VectorTableBuilder(table_name: str, metadata: MetaData | None = None)

Bases: object

Builder class for creating MatrixOne vector tables with SQLAlchemy.

This class provides a fluent interface for building vector tables with proper column definitions, indexes, and constraints. It’s designed to work seamlessly with MatrixOne’s vector capabilities and SQLAlchemy.

Key Features:

• Fluent method chaining for table definition

• Support for all MatrixOne column types including vectors

• Automatic vector index creation

• Constraint and foreign key support

• Integration with SQLAlchemy metadata

Supported Column Types: - Standard types: Integer, String, Text, DateTime, etc. - Vector types: Vectorf32, Vectorf64 with configurable dimensions - MatrixOne-specific types: JSON, BLOB variants

Usage Examples

# Create a simple vector table
builder = VectorTableBuilder('documents')
table = (builder
        .add_int_column('id', primary_key=True)
        .add_string_column('title', length=255)
        .add_text_column('content')
        .add_vector_column('embedding', Vectorf32(384))
        .build())

# Create a complex table with indexes
builder = VectorTableBuilder('products')
table = (builder
        .add_bigint_column('id', primary_key=True)
        .add_string_column('name', length=100)
        .add_numeric_column('price', precision=10, scale=2)
        .add_vector_column('features', Vectorf64(512))
        .add_vector_index('idx_features', 'features', 'ivfflat', lists=100)
        .build())

Note: This builder is primarily used internally by the Client’s table creation methods, but can be used directly for advanced use cases.

__init__(table_name: str, metadata: MetaData | None = None)

Initialize the table builder.

Parameters:

• table_name – Name of the table to create

• metadata – SQLAlchemy metadata object

add_column(name: str, type_, **kwargs)

Add a column to the table.

add_int_column(name: str, primary_key: bool = False, **kwargs)

Add an integer column.

add_bigint_column(name: str, primary_key: bool = False, **kwargs)

Add a bigint column.

add_string_column(name: str, length: int = 255, **kwargs)

Add a string column.

add_text_column(name: str, **kwargs)

Add a text column.

add_json_column(name: str, **kwargs)

Add a JSON column.

add_vector_column(name: str, dimension: int, precision: str = 'f32', **kwargs)

Add a vector column.

add_vecf32_column(name: str, dimension: int, **kwargs)

Add a vecf32 column.

add_vecf64_column(name: str, dimension: int, **kwargs)

Add a vecf64 column.

add_smallint_column(name: str, primary_key: bool = False, **kwargs)

Add a smallint column.

add_tinyint_column(name: str, primary_key: bool = False, **kwargs)

Add a tinyint column.

add_numeric_column(name: str, column_type: str, precision: int | None = None, scale: int | None = None, **kwargs)

Add a numeric column (float, double, decimal, numeric).

add_datetime_column(name: str, column_type: str, **kwargs)

Add a datetime column (date, datetime, timestamp, time, year).

add_boolean_column(name: str, **kwargs)

Add a boolean column.

add_binary_column(name: str, column_type: str, **kwargs)

Add a binary column (blob, longblob, mediumblob, tinyblob, binary, varbinary).

add_enum_column(name: str, column_type: str, values: list, **kwargs)

Add an enum or set column.

add_index(columns: str | List[str], name: str | None = None, **kwargs)

Add an index to the table.

add_primary_key(columns: str | List[str])

Add a primary key constraint.

add_foreign_key(columns: str | List[str], ref_table: str, ref_columns: str | List[str], name: str | None = None)

Add a foreign key constraint.

build() → Table

Build and return the SQLAlchemy Table object.