Tools

Full-text and semantic search across a media archive on Elasticsearch

An open look at how publishers use Elasticsearch for full-text and semantic search across articles and archives, with The Guardian and DC Thomson as examples

Our clients

Clients and partners

Capital Group
FSK Group
SMLT
Tochno
Dogma
Sber City
FM Logistic
Danone
+10clients · View cases →

This is an overview of public practices, not a KT.Team case study. We gathered publicly documented examples and documentation to show the business results media companies get from content search on Elasticsearch.

Problem: the archive exists, but it does not generate views

Over the years, publishers accumulate tens or hundreds of thousands of articles. But if a reader cannot find relevant material within two or three clicks, the archive does not serve the business: the user leaves after the first page, page depth falls, and ad impressions and the subscription funnel receive less traffic. The metrics that show this are bounce rate, pages per session, and dwell time (time until returning to results). Full-text word search helps only partially here: the query "how to pay less taxes" does not find an article titled "optimization of the fiscal burden" because the words do not match.

What Elasticsearch provides at the search layer

Historically, Elasticsearch covers classic full-text search - morphology, inverted index, BM25 ranking, facets by section, authors, and dates. On top of that, Elastic has added a semantic layer. According to the documentation, the easiest way to enable it is with the `semantic_text` field type: it automatically builds embeddings during indexing and querying, without manual inference pipeline setup. Under the hood, two approaches are available - dense vectors (dense vector, kNN nearest-neighbor search) and Elastic's own sparse model ELSER (Elastic Learned Sparse Encoder), which expands text with a set of weighted related terms. The sparse option is valuable for media because the result is explainable: you can see which terms produced the match (Elastic Docs: Semantic search, Elastic Docs: Semantic search with ELSER).

The key technique is hybrid search: lexical and semantic results are combined using reciprocal rank fusion (RRF). Keyword precision (names, brands, quotes) is preserved, while the semantic layer pulls in related materials even when the wording differs. For an archive, that means it finds what the person meant.

Navigation and recommendations on top of the same index

The same index powers not only the search bar, but also the related articles and read also blocks. Elasticsearch has the More Like This query for this, and with vectors it can find semantically similar documents using the embedding of the current article. That turns every page into an entry point to the archive: a reader comes from social media to one article and gets a relevant next-step chain.

Open examples from the industry

DC Thomson - a British media group (The Press and Journal, Energy Voice, and other publications). After moving to Elasticsearch, the company recorded a 40% increase in page views, a lower bounce rate, and more pages per session; the result was visible within two weeks. Separately, the group used the related articles block as a monetization tool on paywalled sites: by adding tracking to recommendation links, the team showed leadership a measurable contribution to engagement. According to the editorial team, personalized real-time content delivery is critical for the subscription model (Elastic Customers: DC Thomson).

The Guardian uses the Elastic Stack more broadly for analytics as well (the Ophan system processes around 40 million documents per day and is used by the editorial and SEO teams), as well as for the Content API and fast search across the photo archive. This is an example of how one search layer serves both content navigation and the editorial workflows around it (Elastic Customers: The Guardian).

An additional technical reference point is case studies of hybrid search deployments (semantic plus keywords), where the mixed model balances precision and recall across large content catalogs (Search relevance case study, chandangaurav.com).

An approach without rewriting the core

An important long-term ownership detail: Elasticsearch is a mature international search standard, not a custom-built solution. It is better to keep ranking business logic, freshness or subscription boosting, and A/B tests for results in a separate service near the cluster rather than hard-coding them into Elasticsearch forks. That keeps the search layer upgradable and transferable: it can be handed to another team or contractor without a rewrite. Result quality should be measured not by eye, but with judgment lists - labeled sets of "query -> relevant documents" used to calculate metrics before and after changes (Elasticsearch Labs: Judgment lists).

Business process takeaway

Search and recommendations in Elasticsearch reshape how content is consumed: instead of one page read, users follow a chain of relevant paths through the archive. This directly improves media KPIs - page depth and session length - and therefore ad impressions and subscriber retention (DC Thomson example: +40% in views, lower bounce rate). To keep the effect controlled rather than one-off, the search layer should be moved into a separate service near the core and relevance should be measured regularly with judgment lists - then improving results becomes a repeatable process, not manual fine-tuning.

Sources

Content archive

Processing

Elasticsearch indexing pipeline
Inverted indexBM25, morphology
near the core, without forking Elasticsearch

Channels and endpoints

Vectors: semantic_text / ELSER, dense kNN
Hybrid query + RRF
Reader query
Archive search bar
Related articles / read alsoMore Like This, kNN
Relevance analyticsjudgment lists
More page views, longer sessions, lower bounce rate
Ranking business-logic serviceFreshness/subscription boosting, A/B

Which business process it improves

Search and recommendations in Elasticsearch turn one read page into a chain of archive visits, increasing page depth and session length (DC Thomson example: +40% in views, lower bounce rate). To make the effect repeatable, the search layer is moved into a separate service near the core and relevance is measured with judgment lists.

Discuss Full-Text and Semantic Search for...

Send via: