Search Concepts and Configuration
A Search Profile defines which records from a source Object Type to index and how to turn their fields into searchable text. The named indexes for a Search Profile define reusable query behavior. This page explains the main configuration elements, the choices available for each, and how those choices affect indexed content and search results.
Templates
A Search Profile's required body template selects and organizes the source record information that becomes searchable text. Contextual renders the body for each eligible record and splits it into chunks. An optional, separate header template adds context to each chunk before embedding.
Templates use Handlebars syntax and can reference fields from the source Native Object record. Use {{fieldName}} to insert a field's value. For example, {{title}} inserts the record's title value.
Include information that could help answer likely natural-language queries, and label values so their meaning is clear. Use conditional blocks for optional text:
Title: {{title}}
{{#if description}}Description: {{description}}{{/if}}
{{#if resolution}}Resolution: {{resolution}}{{/if}}Some source records store another record's ID in a field such as ownerId. Including {{ownerId}} in a template adds the ID itself, not the related record's name or description. To make those details searchable, include a field that already holds them on the source record, or add and maintain such a field before indexing.
The rendered header is prepended to each chunk before embedding. A short title or category can give each chunk context. Headers consume part of the chunk size, so a long header leaves less room for the body.
Generate from schema creates a starting body template from renderable top-level fields. It does not establish which fields will produce good matches. Review the result for relevance and readability.
The console preview shows the rendered header and body for one sample record, or placeholders when no sample is available. Chunking happens during indexing; the preview does not show chunks or validate all source records.
Chunking
A document is the text derived from a source record. A chunk is a portion of that text that can be embedded and retrieved. Long documents can produce multiple chunks that refer to the same source record.
These settings control how rendered text is divided into chunks:
Splitter
plain-text-window
plain-text-window
Size
1,600 characters
100–128,000 characters
Overlap
10%
0–50%
Preserve boundaries
true
true or false
To split text at paragraph, sentence, or word boundaries where possible, leave Strict windows (not recommended) unchecked in the dashboard (preserveBoundaries: true). See the dashboard control map.
Size is measured in characters. Each chunk sent for embedding, including its rendered header and separator, must also fit the embedding model's input token limit. The indexer reserves space for the header and separator within the character budget before splitting the body.
Overlap repeats some body text between adjacent chunks to retain context around a split. With boundary preservation enabled, the indexer considers paragraph, sentence, and word boundaries. It can fall back to split within a sentence or word to stay within the chunk size limit when necessary. Chunk lengths and actual overlap can therefore differ from fixed windows.
Start with the defaults. Change chunking when retrieved passages omit needed context or combine unrelated topics. An empty rendered body produces no chunks.
Embedding
An embedding represents text as a vector used for semantic comparison. Contextual embeds indexed text and the query using the active index's embedding configuration.
Model
The trained AI system that converts text into vectors representing its meaning.
voyage-4
voyage-4-lite, voyage-4, voyage-4-large
Dimensions
The number of numeric values in each embedding vector.
1,024
512, 1,024, 2,048
A model's name (such as "lite" or "large") does not describe how well it will retrieve relevant content from your records. Before changing embedding models, compare search results for questions users are likely to ask.
Changing dimensions changes the vector configuration and requires rebuilding. The dashboard saves the change without a rebuild confirmation prompt. After saving, inspect the Index panel for progress. See Change an Existing Profile and Keep Search Up to Date.
Projections
Projections let a named Search Index narrow searches by structured values from source records, such as category or rating. During indexing, an active projection evaluates an expression against each record to produce a typed attribute. A named index can use that attribute in a static filter, and its value, when present, is returned with search evidence. A projection does not add text to the template; include the information in a template too if you want it to be searchable as text.
Each projection has a unique alias, an expression language, an expression, and a result type. Aliases start with a lowercase letter and contain lowercase letters, numbers, or underscores, up to 63 characters.
Language
jsonpath, jsonata
Type
string, number, integer, boolean, datetime, string[], number[], integer[]
Retired
Optional true to stop writing and returning the projection while keeping its alias and type reserved
For example, suppose a profile indexes equipment records for pumps, valves, and motors. This projection extracts each record's category:
You could configure a named index called pumps with a static filter of category = "Pumps". A search request with indexId: "pumps" then uses that filter to retrieve pump records. The profile still indexes the other categories, and searches that do not select pumps do not use its filter. The filter value is saved in the named index; the search request supplies the index ID, not a category value.
Preview the expression against real records, including missing fields and unexpected values. The indexer converts results to the declared type; absent or unconvertible values can be omitted. A projection is not a substitute for source-data validation.
Projection Types and Stored Columns
The projection type specifies how the expression's result is converted. Projection values are stored in typed columns in the indexed data shared by a profile's named Search Indexes. Each column keeps its type once created. Filters and returned projection values use the projection alias, such as category; the corresponding stored column is named data.category.
string, string[]
text
Any value, converted to text
number, number[]
floating-point number (float)
Whole and fractional numbers
integer, integer[]
integer (int)
Whole numbers only; a fraction, or a whole number too large to store exactly, skips the record
boolean
boolean
true, false, and the strings "true" and "false"
datetime
datetime
Dates and timestamps, stored as ISO 8601
Choose number unless every value is a whole number and you want an integer column. Neither type rounds a value to make it fit: an integer projection that produces 2.5 skips that record and reports it. Because a JSONata expression's result is known only when it runs, an integer projection over computed values, such as an average, can skip records even when the source fields are whole numbers.
Indexes created before explicit column types can hold columns whose type came from the first value written. A number projection whose column holds integers keeps working for whole values. A fraction is not rounded: the indexer skips that record and reports it (see Skipped Records). A datetime projection whose column holds text keeps working. The dashboard shows each column's stored type under Stored columns.
Some conversions happen before storage:
An expression that returns
null, or matches nothing, produces no value. The indexer omits the attribute rather than storing zero or an empty value.Values are converted to the declared projection type, so a numeric string such as
"12"becomes12for anumberprojection. A value that cannot convert is omitted.A JSONata expression can return a different kind of value than the source field holds. For example,
$count(tags)returns a number from an array field. Choose the type for what the expression returns. The source field's schema alone does not establish it.
The preview evaluates one sample record. It cannot show that every future value fits the stored column.
Add, Rename, Remove, or Retire a Projection
Changing projections only, with source, template, chunking, and embedding unchanged, updates the existing index in place. A background repair reprocesses records and reuses their embeddings, so unchanged text needs no new embedding.
Add a projection
The new column is created with its declared type before any record is written.
Change an expression, or a type the stored column can hold
The repair rewrites values in place.
Rename a projection
The new alias is a new column. The old column becomes a removed column, described below.
Change to a type the stored column cannot hold, such as text to number
The save does not apply in place. Choose a compatible type or a new alias, or save with a rebuild into a fresh index.
Retire a projection
Writes, filters, and results stop using it right away. The projection stays in the profile, marked "retired": true, and appears in a collapsed Retired group. Stored values stay on each record until that record is next rewritten.
Re-enable a retired projection
Writes, filters, and results use it again, and the repair rewrites its values. It must keep a type the stored column can hold.
Remove a projection
The same as retiring, except the projection leaves the profile. Its stored column shows as removed.
Retiring or removing a projection never frees its column in the current index. The name and type stay reserved there, so re-enabling a retired projection, or reusing a removed alias, needs a compatible type. To change a type, use a new alias, or rebuild. A rebuild starts a fresh index that declares only the active projections; retired projections stay retired in the profile and get no column there.
Indexes
A named Search Index is a query preset within a profile. Multiple presets can use the same indexed records with different retrieval settings.
Mode
hybrid
Semantic plus keyword retrieval; semantic uses vector similarity.
Top K
10
Retrieval candidate limit, from 1 to 100.
One result per document
true
Group retrieval by source document.
Static filters
None
Restrict candidates using projected fields.
Reranking
Disabled
Optionally reorder retrieved candidates.
To use a named preset, include its ID in the indexId field of your search query request. If you omit indexId, the search uses semantic mode, Top K 10, no document grouping, no preset filters, and no reranking. You can override Top K by including topK in the request.
Top K sets the maximum number of retrieval candidates. A query can return fewer results. Filtering, grouping, and source-record availability affect the result count.
Named index settings apply when you run a search. Saving changes to a named index's retrieval settings does not trigger a rebuild or change stored embeddings. After you save, the next query that selects the named index uses the new values.
Static Filters
Static filters restrict a named index's candidates using projection aliases. All configured static filters must match.
eq, not-eq
Compare a projected value with the supplied value.
lt, lte, gt, gte
Compare ordered values. Use a value compatible with the projected type.
in, not-in
Supply an array of candidate values.
contains
Test membership in an array-valued field with a single value.
contains-any
Supply an array of values to match against an array-valued field.
glob, regex
Supply a string pattern; regex must be a valid regular expression.
Source eligibility and static filters apply at different stages:
Source eligibility selects which source records belong in the profile.
Static filters restrict searches that use a particular named index.
Reranking
Reranking scores retrieved candidates again against the query and reorders them.
To use reranking, enable it on a named Search Index and select a rerank preset: a saved configuration that identifies the model used for reranking. Available presets depend on your tenant's configuration. Set Top N explicitly to limit how many of the highest-scoring candidates are retained after reranking. Top N must not exceed the configured Top K.
The default scope is document. Each document is reranked once, using the text of its highest-ranked retrieved hit rather than the full source record. The document's other retrieved hits follow it and share its rerank score. Because Top N counts documents in this scope, turning grouping off can return more hit rows than Top N.
With chunk scope, reranking scores each retrieved text chunk separately, including chunks from the same source record. Choose a scope that matches whether the reader needs distinct records or individual passages. Compare it with reranking disabled using the same queries.
Last updated
Was this helpful?