Search
Search
Prospect (smeltery::prospect) is full-text search for models. A model declares which columns are searched (with
weights), filtered and sorted; a search returns a Page of hits that hold real models, ranked by relevance, with
highlights. The database driver uses the database's own full-text search: SQLite FTS5, PostgreSQL tsvector with a
GIN index, MySQL / MariaDB FULLTEXT. The database keeps the index current itself (triggers, a generated column, the
FULLTEXT index), so every write is searchable at once, raw SQL and bulk updates included, and no extra server runs.
Making a model searchable#
impl Searchable next to the model's entity, register the model in bootstrap/app.rs, and add the index with a
migration:
use smeltery::db::prelude::*;
use smeltery::prospect::{IndexSpec, Searchable, Weight};
#[sea_orm::model]
#[derive(Clone, Debug, PartialEq, DeriveEntityModel, Serialize, Deserialize)]
#[sea_orm(table_name = "posts")]
pub struct Model {
#[sea_orm(primary_key)]
pub id: i64,
pub title: String,
pub body: String,
pub user_id: i64,
pub team_id: i64,
pub published: bool,
pub created_at: Option<DateTimeUtc>,
}
impl ActiveModelBehavior for ActiveModel {}
impl Searchable for Model {
fn index(i: &mut IndexSpec) {
i.text("title").weight(Weight::A); // searched, ranked highest
i.text("body"); // Weight::B
i.filter("user_id"); // where_eq / where_in / where_not_in / where_between on it
i.sort("created_at"); // order_by on it (relevance is the default order)
i.only_when("published"); // rows with false are not found
i.scoped_by("team_id"); // every search names a team
}
}
use smeltery::prelude::*;
pub fn build(app: AppBuilder) -> AppBuilder {
app.prospect(|p| {
p.model::<Post>();
})
}
IndexSpec call |
Means |
|---|---|
i.text(column), .weight(Weight::A) |
a searched string column; weights A (highest) to D, default B; the order of the calls is the order of the index's columns |
i.filter(column) |
an integer, bool, string or date-time column searches may filter on |
i.sort(column) |
a column searches may order by |
i.only_when(column) |
a bool column: rows where it is false are not found |
i.scoped_by(column) |
every search must call within(value), or across_scopes() explicitly; the column is also a filter column |
i.language(Language::English) |
English stemming on SQLite and PostgreSQL (forging finds forge); the default Language::Simple compares words as they are |
i.name(name) |
the memory engine's index name (default: the table's); the database index is always <table>_search |
The spec is checked when the app builds: every name must be a column of the model, text columns strings,
only_when a bool, at least one text, and the model needs one integer primary key (t.id()). A mistake, a model
registered twice or an unknown PROSPECT_DRIVER stops the build with the table and the column.
The migration names the same text columns, weights and language:
use smeltery::db::migration::{Migration, Schema};
use smeltery::prospect::migration::{Language, SearchIndex, Weight};
use smeltery::Result;
pub struct CreatePostsTable;
impl Migration for CreatePostsTable {
fn name(&self) -> &'static str {
"2026_10_05_120000_create_posts_table"
}
async fn up(&self, schema: &Schema) -> Result<()> {
schema
.create("posts", |t| {
t.id();
t.string("title");
t.text("body");
t.big_integer("user_id");
t.big_integer("team_id");
t.boolean("published").default(false);
t.timestamps();
})
.await?;
SearchIndex::on("posts")
.text("title", Weight::A)
.text("body", Weight::B)
.language(Language::Simple)
.create(schema)
.await
}
async fn down(&self, schema: &Schema) -> Result<()> {
SearchIndex::on("posts").drop(schema).await?;
schema.drop_if_exists("posts").await
}
}
| Backend | SearchIndex::create writes |
drop removes |
|---|---|---|
| SQLite | the FTS5 table posts_search (external content over posts, keyed by id), the triggers posts_search_ai, _ad, _au that keep it current, and a rebuild that indexes the rows already there |
the triggers and the table |
| PostgreSQL | the stored generated column search_vector (tsvector, weighted) and the GIN index posts_search_index |
the index and the column |
| MySQL / MariaDB | the FULLTEXT index posts_search over the text columns |
the index |
SearchIndex::on("posts").rebuild(schema) drops and creates it again, for a migration that changes the columns,
weights or language. .key("post_id") names the integer key column when the model's primary key is not id. Adding the generated column on PostgreSQL
rewrites the table, and the first FULLTEXT index of a MySQL table rebuilds it: run such a migration while the table
is small or during maintenance. The first search of a model checks its index: on SQLite the FTS5 table's text
columns in order, its table and key (content, content_rowid) and the three triggers; on PostgreSQL the columns,
weights and language of the search_vector expression (read from information_schema, so the app's database user
owns the table) and its GIN index; on MySQL the FULLTEXT index's columns in
order. A missing or different index is logged once (ERROR) and every search of that model answers a 500 whose
message names the migration to write; serve logs it at start, and prospect:status shows it.
Searching#
use smeltery::prelude::*;
use smeltery::prospect::{Direction, Hit, Prospect};
#[derive(serde::Deserialize)]
struct SearchForm {
#[serde(default)]
q: String,
}
async fn index(prospect: Prospect, Query(s): Query<SearchForm>, page: PageQuery) -> Result<Json<Page<Hit<Post>>>> {
let team_id = 1; // from the signed-in user
let posts = Post::search(&prospect, &s.q)
.within(team_id) // the scope: required because of scoped_by
.where_eq("user_id", 7) // declared filter columns only
.order_by_relevance() // the default; or .order_by("created_at", Direction::Desc)
.highlight(["title", "body"])
.paginate(page) // ?page=&per_page=, bounded
.await?;
Ok(Json(posts))
}
pub fn routes(r: &mut Router) {
r.get("/posts", index).middleware("throttle:60,1");
}
Post::search(&prospect, text) starts a search (smeltery::prelude brings the Searchable trait into scope). The
builder:
| Call | Means |
|---|---|
within(value) / across_scopes() |
the scope value, or every scope (explicitly) |
where_eq(column, value), where_in(column, values), where_not_in(column, values), where_between(column, low, high) |
conditions on filter columns; values of the column's type (i64 and smaller integers, bool, strings, DateTimeUtc); where_in and where_not_in take at most 100 values |
order_by(column, Direction::Asc / Desc), order_by_relevance() |
a sort column, or relevance (the default); ties by key, newest first |
highlight(columns) |
highlights of at most 4 text columns |
query(\|select\| select.filter(…)) |
a SeaORM refinement of the same SQL statement ("published or mine"); database driver only |
paginate(page) |
a Page<Hit<M>>: one page and the total (two statements) |
get(limit), keys(limit), count() |
the first hits, their keys, the number of matches |
An undeclared column, a value of the wrong type, a bound passed, or a scoped model searched without within /
across_scopes makes the call that runs the search return an error (a 500) before any query;
ProspectError::of(&error) reads which.
The search text. User text is never search syntax. It is trimmed and cut at PROSPECT_MAX_QUERY_LENGTH
characters (200), split into terms on everything that is not a letter or a digit, lower-cased, and at most 16 terms are
kept. Every term must match; the last one also matches as a prefix when it has two characters or more (forg finds
forge), so a search-as-you-type box works. Quotes, *, :, -, parentheses and words like AND, OR, NOT,
NEAR are plain text. A text without terms (empty, or punctuation only) has no text condition: the filtered rows come
newest key first, so an index page with an empty search box keeps working.
Results. A Hit<Post> derefs to the post (hit.title) and has score (Option<f64>, higher is better, None
without terms; scores compare the hits of one search) and highlights. It serializes as the model's fields plus
_score and _highlights, so an Alloy page reads it directly. A highlight is a list of text segments,
{"text": …, "matched": true / false} (Segment), never HTML; render each one escaped and wrap the matched ones.
A Mold template reads struct fields, which release builds compile to Rust field access, so the handler hands it the
segments in a field (hit.highlights.get("title").map(|h| h.segments().to_vec()).unwrap_or_default(), a
Vec<Segment>, next to the record from hit.into_model()), and the template loops over them (the empty comment
keeps @endfor from touching the word before it):
@for(row in posts.items)
<h2>@for(s in row.title)@if(s.matched)<mark>{{ s.text }}</mark>@else{{ s.text }}@endif{{-- --}}@endfor</h2>
@endfor
In React and Vue, render segments as text nodes inside <mark>. A column of at most 2,000 characters comes back whole
with every match marked; above that it is a snippet around the matches (SQLite: 32 tokens; PostgreSQL: two fragments
of 10 to 30 words from the first 20,000 characters). The memory engine and MySQL highlight the first 20,000
characters in Rust and show a window of about 30 words around the first match of a text longer than 60 words, with
… around it. Characters U+E000 and U+E001 never appear in a highlight.
Pages. paginate takes a PageQuery (see Pagination); per_page is at most
PROSPECT_MAX_PER_PAGE (100), and get takes a limit of 1 to that value. A search ranks and counts every matching
row, so its cost grows with the number of matches: on large tables keep a throttle: on search routes.
Drivers#
| SQLite | PostgreSQL | MySQL / MariaDB | |
|---|---|---|---|
| Index | FTS5 table + triggers | generated tsvector + GIN |
FULLTEXT |
| Ranking | bm25 with column weights 10 / 5 / 2 / 1 |
ts_rank_cd with weights A-D |
MATCH … AGAINST relevance, no column weights |
| Stemming | Language::English (porter) |
Language::English (english) |
none |
| Left out of a search | nothing | nothing | words shorter than the server's innodb_ft_min_token_size (read once; 3 when it cannot be read) and InnoDB's default stopwords (prospect::MYSQL_STOPWORDS: the, a, for, with …); a text with no other word searches like an empty one. The default list is assumed: a server with its own innodb_ft_server_stopword_table or innodb_ft_enable_stopword=OFF differs (a required custom stopword finds nothing) |
| Highlights | FTS5 highlight / snippet |
ts_headline, on the page's rows only |
in Rust |
PROSPECT_DRIVER=memory is an in-process engine for tests: documents hold the declared columns only and are written
by model events (Record writes; see Model events), searched with the same text rules, and every hit
is loaded from the database with the scope, only_when and filters applied again in SQL, so a stale document never
shows a record the database would not return. It does not take query(…), and its string filter values are letters,
digits and _ . : @ - only (1 to 128 characters).
Settings and commands#
.env |
Default | Meaning |
|---|---|---|
PROSPECT_DRIVER |
database |
database or memory; anything else stops the build |
PROSPECT_MAX_QUERY_LENGTH |
200 |
characters of search text used (1 to 1,000) |
PROSPECT_MAX_PER_PAGE |
100 |
the largest page or get limit (1 to 1,000) |
PROSPECT_BATCH |
500 |
rows per chunk when Prospect::import fills the memory engine (1 to 10,000) |
Values out of range are clamped with a warning. app.prospect_with(ProspectSettings::from_env().max_per_page(50), …)
sets them in code.
| Command | Does |
|---|---|
smeltery prospect:status |
the driver; per searchable model its rows, text columns and whether its index is there |
smeltery prospect:import [table…] |
SQLite: rebuild the FTS5 index from the table; PostgreSQL / MySQL keep theirs current (nothing to do) |
smeltery prospect:flush table |
remove a model's documents from the memory engine (refused by the database driver: its index follows the table) |
In code: prospect.import::<Post>(), prospect.sync::<Post>(keys) (for writes outside Record with the memory
engine), prospect.flush::<Post>(), and prospect.paused(async { … }), which runs a block with model listeners
switched off.
The search text is never logged; at DEBUG a search logs its table and the number of terms and filters.
Testing search#
With the database driver, searches work in tests on in-memory SQLite (TestApp) like in the app: FTS5 is part of
the bundled SQLite. prospect::testing::fake(&app) switches a TestApp to the memory engine and records what is
indexed:
use smeltery::db::prelude::*;
use smeltery::prospect::{ProspectExt as _, testing};
use smeltery::testing::TestApp;
fn drafts_are_not_indexed() {
let app = TestApp::new(|b| b.prospect(|p| { p.model::<Post>(); }));
let fake = testing::fake(&app);
let db = app.db();
let draft = app
.block_on(Post::create(&db, post::ActiveModel {
title: Set("Draft".into()),
published: Set(false),
..Default::default()
}))
.unwrap();
fake.assert_not_indexed::<Post>(draft.id);
fake.assert_synced_times::<Post>(1);
}