Smeltery
Get started

Search

v0.1

Search

On this page

Prospect (smeltery::prospect) is full-text search for models. A model declares which columns are searched (with weights), filtered and sorted; a search returns a Page of hits that hold real models, ranked by relevance, with highlights. The database driver uses the database's own full-text search: SQLite FTS5, PostgreSQL tsvector with a GIN index, MySQL / MariaDB FULLTEXT. The database keeps the index current itself (triggers, a generated column, the FULLTEXT index), so every write is searchable at once, raw SQL and bulk updates included, and no extra server runs.

Making a model searchable#

impl Searchable next to the model's entity, register the model in bootstrap/app.rs, and add the index with a migration:

rust
use smeltery::db::prelude::*;
use smeltery::prospect::{IndexSpec, Searchable, Weight};

#[sea_orm::model]
#[derive(Clone, Debug, PartialEq, DeriveEntityModel, Serialize, Deserialize)]
#[sea_orm(table_name = "posts")]
pub struct Model {
    #[sea_orm(primary_key)]
    pub id: i64,
    pub title: String,
    pub body: String,
    pub user_id: i64,
    pub team_id: i64,
    pub published: bool,
    pub created_at: Option<DateTimeUtc>,
}

impl ActiveModelBehavior for ActiveModel {}

impl Searchable for Model {
    fn index(i: &mut IndexSpec) {
        i.text("title").weight(Weight::A); // searched, ranked highest
        i.text("body"); // Weight::B
        i.filter("user_id"); // where_eq / where_in / where_not_in / where_between on it
        i.sort("created_at"); // order_by on it (relevance is the default order)
        i.only_when("published"); // rows with false are not found
        i.scoped_by("team_id"); // every search names a team
    }
}
use smeltery::prelude::*;

pub fn build(app: AppBuilder) -> AppBuilder {
    app.prospect(|p| {
        p.model::<Post>();
    })
}
IndexSpec call Means
i.text(column), .weight(Weight::A) a searched string column; weights A (highest) to D, default B; the order of the calls is the order of the index's columns
i.filter(column) an integer, bool, string or date-time column searches may filter on
i.sort(column) a column searches may order by
i.only_when(column) a bool column: rows where it is false are not found
i.scoped_by(column) every search must call within(value), or across_scopes() explicitly; the column is also a filter column
i.language(Language::English) English stemming on SQLite and PostgreSQL (forging finds forge); the default Language::Simple compares words as they are
i.name(name) the memory engine's index name (default: the table's); the database index is always <table>_search

The spec is checked when the app builds: every name must be a column of the model, text columns strings, only_when a bool, at least one text, and the model needs one integer primary key (t.id()). A mistake, a model registered twice or an unknown PROSPECT_DRIVER stops the build with the table and the column.

The migration names the same text columns, weights and language:

rust
use smeltery::db::migration::{Migration, Schema};
use smeltery::prospect::migration::{Language, SearchIndex, Weight};
use smeltery::Result;

pub struct CreatePostsTable;

impl Migration for CreatePostsTable {
    fn name(&self) -> &'static str {
        "2026_10_05_120000_create_posts_table"
    }

    async fn up(&self, schema: &Schema) -> Result<()> {
        schema
            .create("posts", |t| {
                t.id();
                t.string("title");
                t.text("body");
                t.big_integer("user_id");
                t.big_integer("team_id");
                t.boolean("published").default(false);
                t.timestamps();
            })
            .await?;
        SearchIndex::on("posts")
            .text("title", Weight::A)
            .text("body", Weight::B)
            .language(Language::Simple)
            .create(schema)
            .await
    }

    async fn down(&self, schema: &Schema) -> Result<()> {
        SearchIndex::on("posts").drop(schema).await?;
        schema.drop_if_exists("posts").await
    }
}
Backend SearchIndex::create writes drop removes
SQLite the FTS5 table posts_search (external content over posts, keyed by id), the triggers posts_search_ai, _ad, _au that keep it current, and a rebuild that indexes the rows already there the triggers and the table
PostgreSQL the stored generated column search_vector (tsvector, weighted) and the GIN index posts_search_index the index and the column
MySQL / MariaDB the FULLTEXT index posts_search over the text columns the index

SearchIndex::on("posts").rebuild(schema) drops and creates it again, for a migration that changes the columns, weights or language. .key("post_id") names the integer key column when the model's primary key is not id. Adding the generated column on PostgreSQL rewrites the table, and the first FULLTEXT index of a MySQL table rebuilds it: run such a migration while the table is small or during maintenance. The first search of a model checks its index: on SQLite the FTS5 table's text columns in order, its table and key (content, content_rowid) and the three triggers; on PostgreSQL the columns, weights and language of the search_vector expression (read from information_schema, so the app's database user owns the table) and its GIN index; on MySQL the FULLTEXT index's columns in order. A missing or different index is logged once (ERROR) and every search of that model answers a 500 whose message names the migration to write; serve logs it at start, and prospect:status shows it.

Searching#

rust
use smeltery::prelude::*;
use smeltery::prospect::{Direction, Hit, Prospect};

#[derive(serde::Deserialize)]
struct SearchForm {
    #[serde(default)]
    q: String,
}

async fn index(prospect: Prospect, Query(s): Query<SearchForm>, page: PageQuery) -> Result<Json<Page<Hit<Post>>>> {
    let team_id = 1; // from the signed-in user
    let posts = Post::search(&prospect, &s.q)
        .within(team_id) // the scope: required because of scoped_by
        .where_eq("user_id", 7) // declared filter columns only
        .order_by_relevance() // the default; or .order_by("created_at", Direction::Desc)
        .highlight(["title", "body"])
        .paginate(page) // ?page=&per_page=, bounded
        .await?;
    Ok(Json(posts))
}

pub fn routes(r: &mut Router) {
    r.get("/posts", index).middleware("throttle:60,1");
}

Post::search(&prospect, text) starts a search (smeltery::prelude brings the Searchable trait into scope). The builder:

Call Means
within(value) / across_scopes() the scope value, or every scope (explicitly)
where_eq(column, value), where_in(column, values), where_not_in(column, values), where_between(column, low, high) conditions on filter columns; values of the column's type (i64 and smaller integers, bool, strings, DateTimeUtc); where_in and where_not_in take at most 100 values
order_by(column, Direction::Asc / Desc), order_by_relevance() a sort column, or relevance (the default); ties by key, newest first
highlight(columns) highlights of at most 4 text columns
query(\|select\| select.filter(…)) a SeaORM refinement of the same SQL statement ("published or mine"); database driver only
paginate(page) a Page<Hit<M>>: one page and the total (two statements)
get(limit), keys(limit), count() the first hits, their keys, the number of matches

An undeclared column, a value of the wrong type, a bound passed, or a scoped model searched without within / across_scopes makes the call that runs the search return an error (a 500) before any query; ProspectError::of(&error) reads which.

The search text. User text is never search syntax. It is trimmed and cut at PROSPECT_MAX_QUERY_LENGTH characters (200), split into terms on everything that is not a letter or a digit, lower-cased, and at most 16 terms are kept. Every term must match; the last one also matches as a prefix when it has two characters or more (forg finds forge), so a search-as-you-type box works. Quotes, *, :, -, parentheses and words like AND, OR, NOT, NEAR are plain text. A text without terms (empty, or punctuation only) has no text condition: the filtered rows come newest key first, so an index page with an empty search box keeps working.

Results. A Hit<Post> derefs to the post (hit.title) and has score (Option<f64>, higher is better, None without terms; scores compare the hits of one search) and highlights. It serializes as the model's fields plus _score and _highlights, so an Alloy page reads it directly. A highlight is a list of text segments, {"text": …, "matched": true / false} (Segment), never HTML; render each one escaped and wrap the matched ones. A Mold template reads struct fields, which release builds compile to Rust field access, so the handler hands it the segments in a field (hit.highlights.get("title").map(|h| h.segments().to_vec()).unwrap_or_default(), a Vec<Segment>, next to the record from hit.into_model()), and the template loops over them (the empty comment keeps @endfor from touching the word before it):

text
@for(row in posts.items)
  <h2>@for(s in row.title)@if(s.matched)<mark>{{ s.text }}</mark>@else{{ s.text }}@endif{{-- --}}@endfor</h2>
@endfor

In React and Vue, render segments as text nodes inside <mark>. A column of at most 2,000 characters comes back whole with every match marked; above that it is a snippet around the matches (SQLite: 32 tokens; PostgreSQL: two fragments of 10 to 30 words from the first 20,000 characters). The memory engine and MySQL highlight the first 20,000 characters in Rust and show a window of about 30 words around the first match of a text longer than 60 words, with … around it. Characters U+E000 and U+E001 never appear in a highlight.

Pages. paginate takes a PageQuery (see Pagination); per_page is at most PROSPECT_MAX_PER_PAGE (100), and get takes a limit of 1 to that value. A search ranks and counts every matching row, so its cost grows with the number of matches: on large tables keep a throttle: on search routes.

Drivers#

SQLite PostgreSQL MySQL / MariaDB
Index FTS5 table + triggers generated tsvector + GIN FULLTEXT
Ranking bm25 with column weights 10 / 5 / 2 / 1 ts_rank_cd with weights A-D MATCH … AGAINST relevance, no column weights
Stemming Language::English (porter) Language::English (english) none
Left out of a search nothing nothing words shorter than the server's innodb_ft_min_token_size (read once; 3 when it cannot be read) and InnoDB's default stopwords (prospect::MYSQL_STOPWORDS: the, a, for, with …); a text with no other word searches like an empty one. The default list is assumed: a server with its own innodb_ft_server_stopword_table or innodb_ft_enable_stopword=OFF differs (a required custom stopword finds nothing)
Highlights FTS5 highlight / snippet ts_headline, on the page's rows only in Rust

PROSPECT_DRIVER=memory is an in-process engine for tests: documents hold the declared columns only and are written by model events (Record writes; see Model events), searched with the same text rules, and every hit is loaded from the database with the scope, only_when and filters applied again in SQL, so a stale document never shows a record the database would not return. It does not take query(…), and its string filter values are letters, digits and _ . : @ - only (1 to 128 characters).

Settings and commands#

.env Default Meaning
PROSPECT_DRIVER database database or memory; anything else stops the build
PROSPECT_MAX_QUERY_LENGTH 200 characters of search text used (1 to 1,000)
PROSPECT_MAX_PER_PAGE 100 the largest page or get limit (1 to 1,000)
PROSPECT_BATCH 500 rows per chunk when Prospect::import fills the memory engine (1 to 10,000)

Values out of range are clamped with a warning. app.prospect_with(ProspectSettings::from_env().max_per_page(50), …) sets them in code.

Command Does
smeltery prospect:status the driver; per searchable model its rows, text columns and whether its index is there
smeltery prospect:import [table…] SQLite: rebuild the FTS5 index from the table; PostgreSQL / MySQL keep theirs current (nothing to do)
smeltery prospect:flush table remove a model's documents from the memory engine (refused by the database driver: its index follows the table)

In code: prospect.import::<Post>(), prospect.sync::<Post>(keys) (for writes outside Record with the memory engine), prospect.flush::<Post>(), and prospect.paused(async { … }), which runs a block with model listeners switched off.

The search text is never logged; at DEBUG a search logs its table and the number of terms and filters.

With the database driver, searches work in tests on in-memory SQLite (TestApp) like in the app: FTS5 is part of the bundled SQLite. prospect::testing::fake(&app) switches a TestApp to the memory engine and records what is indexed:

rust
use smeltery::db::prelude::*;
use smeltery::prospect::{ProspectExt as _, testing};
use smeltery::testing::TestApp;

fn drafts_are_not_indexed() {
    let app = TestApp::new(|b| b.prospect(|p| { p.model::<Post>(); }));
    let fake = testing::fake(&app);
    let db = app.db();
    let draft = app
        .block_on(Post::create(&db, post::ActiveModel {
            title: Set("Draft".into()),
            published: Set(false),
            ..Default::default()
        }))
        .unwrap();
    fake.assert_not_indexed::<Post>(draft.id);
    fake.assert_synced_times::<Post>(1);
}

↑↓ moveEnter openEsc close