Curation

Pin or hide hits by ID, attach override tags, and optionally filter hidden hits from the materialized view.

Related: Observability, Troubleshooting → Curation

Curate results by pinning or hiding specific IDs, optionally tagging with override tags, and optionally filtering hidden hits from the materialized view. Purely declarative; encoded as body params only.

Overview

• Pin hits by ID to the top of results (stable first-occurrence order)
• Hide hits by ID (hide-wins when an ID is both pinned and hidden)
• Override tags are optional body-only tags
• Filter flag filter_curated_hits optionally excludes hidden hits from the curated view
• Composes with selection, presets, grouping, pagination; does not alter URL/common params

DSL

Immutable chainers on Relation (copy-on-write). Inputs are normalized (coerced to String, blank dropped), arrays flattened one level, and lists de-duplicated while preserving first occurrence order.

• pin(*ids) — append to pinned list (stable-dedupe)
• hide(*ids) — append to hidden list (set semantics)
• curate(pin: [], hide: [], override_tags: [], filter_curated_hits: nil|true|false) — replace provided keys; omit to retain
• clear_curation — remove all curation state from the relation

State shape on the relation:

• pinned: Array<String>
• hidden: Array<String>
• override_tags: Array<String>
• filter_curated_hits: true | false | nil

Inspect/explain:

• inspect emits a compact token only when non-empty, e.g. curation=p:[p_12,p_34]|h:[p_99]|tags:[homepage]|fch:false
• explain adds a concise curation summary and a conflicts line when overlaps/limits occur

Insert:

# Pin two products to the top and hide one, with an override tag
SearchEngine::Book
  .pin("p_12", "p_34")
  .hide("p_99")
  .curate(override_tags: ["homepage"], filter_curated_hits: false)

# One‑shot
SearchEngine::Book.curate(pin: %w[p_12 p_34], hide: %w[p_99], override_tags: %w[homepage])

Compiler mapping

Curation state maps to Typesense body params and never appears in URL/common params. Empty arrays are omitted; filter_curated_hits is omitted when nil.

State key	Example value	Param key	Encoded value
pinned	["p_1","p_2"]	pinned_hits	"p_1,p_2"
hidden	"p9"	hidden_hits	"p9"
override_tags	["homepage","campaign"]	override_tags	"homepage,campaign"
filter_curated_hits	true	filter_curated_hits	true

• Keys are omitted when arrays are empty or when filter_curated_hits is nil
• Ordering is deterministic; pinned preserves first-occurrence order

Insert:

rel = SearchEngine::Book
        .curate(pin: %w[p_1 p_2], hide: %w[p9], override_tags: %w[homepage], filter_curated_hits: true)
rel.to_typesense_params
# => {
#   q: "*", query_by: "name, description",
#   pinned_hits: "p_1,p_2", hidden_hits: "p9",
#   filter_curated_hits: true, override_tags: "homepage"
# }

Guardrails & errors

Validation is applied after normalization. Overlaps and limits are recorded for explain and observability.

Rules

Rule	Behavior
ID format	SearchEngine.config.curation.id_regex (default /\A[\w-:.]+\z/) applied to curated IDs and override tags
Deduplication	pinned stable-dedupes (first occurrence wins); hidden set-dedupes (first-seen order preserved)
Limits	max_pins (default 50) and max_hidden (default 200) enforced post-normalization
Precedence	When an ID exists in both lists, hide wins (removed from pinned, recorded as conflict)

Errors

Error	When
InvalidCuratedId	Curated ID fails the allowed pattern
CurationLimitExceeded	Pinned or hidden list exceeds configured limit
InvalidOverrideTag	Override tag is blank or fails the allowed pattern

Config example:

SearchEngine.configure do |c|
  c.curation = OpenStruct.new(max_pins: 50, max_hidden: 200, id_regex: /\A[\w\-:\.]+\z/)
end

Multi-search

Per-search independence: each m.add relation carries its own curation keys in its body. Pinned order is preserved; omission rules apply; filter_curated_hits is scoped per entry.

Insert:

res = SearchEngine.multi_search do |m|
  m.add :books, SearchEngine::Book.curate(pin: %w[p_1 p_2])
  m.add :publishers,   SearchEngine::Publisher.curate(hide: %w[b9 b10], filter_curated_hits: true)
end

See also: Multi‑search

Materializers & explain

Materializers reuse the memoized single response and apply curation in-memory.

• Ordering: pins first (declared order, present IDs only), then remainder in original order. Hide-wins.
• Filtering: when filter_curated_hits: true, hidden hits are excluded from iteration and counts.
• Counts: when filtering is on, count reflects the curated view size; exists? follows server totals. To check curated emptiness, use count > 0.
• explain adds a curation summary and a conflicts line.

Explain excerpt:

Curation: pinned=2 hidden=1 filter_curated_hits=false override_tags=[homepage]
Conflicts: [p_1 (both pinned & hidden → hidden)]

Observability

Events are counts/flags only; IDs/tags are redacted. A compact logging subscriber appends a short curation segment to single-search lines and structured JSON fields when present.

• search_engine.curation.compile — once per compile when curation state exists
  • Payload: pinned_count, hidden_count, has_override_tags, filter_curated_hits
• search_engine.curation.conflict — emitted when overlaps or limits are detected; at most once per compile
  • Payload: type (:overlap|:limit_exceeded), count, optional limit

Compact logging (examples; no IDs/tags):

• Text: [se.search] collection=products status=200 duration=12.3ms cu=p:2|h:1|f:false|t:1
• JSON: { "event":"search", "collection":"books", "curation_pinned_count":2, "curation_hidden_count":1, "curation_has_override_tags":true, "curation_filter_flag":false }