Skip to content

Screening keyword highlighting

Project administrators configure two keyword lists, inclusion keywords and exclusion keywords. When a reviewer screens a study on the stage review page, matches for those terms are highlighted inside the PDF shown by the Study Source panel's integrated viewer. The feature is visual assistance only: it never records, suggests or changes a screening decision, and it performs no ranking or classification.

Related issues: #671 (epic), #1366 (highlight keywords from inclusion/exclusion criteria).

Scope and slices

Slice Content Status
A Domain model + pmProject persistence, project-admin endpoint, feature flag, Screening Settings UI, docs Implemented in #3491
B Text layer + highlight engine in StudyPdfViewerComponent, panel/stage-review wiring, ADR-014 addendum Implemented in #3492
Follow-ups Popup viewer parity (protocol extension), whole-document match counts / jump-to-match, diacritic folding, title/abstract highlighting in the detached source window Not started; tracked as follow-up issues

Out of scope by design: OCR for image-only PDFs, automatic include/exclude, AI classification, ranking, any change to screening answers or the decision card.

Flag decision

New flag screeningKeywordHighlighting (default off everywhere; featureFlags.services: [api], web half tsName: screeningKeywordHighlighting). It gates:

  • the Screening Settings "Keyword highlighting" section (web),
  • the PUT api/projects/{projectId}/screening-keywords endpoint ([FeatureGated], 404 while off),
  • the highlight layer in the viewer (web; the viewer itself is additionally behind integratedPdfViewer).

Why flagged: the settings surface would otherwise appear for every project administrator in production while the viewer that consumes it stays behind integratedPdfViewer; the flag keeps the two surfaces coherent and provides a kill switch. Reading the lists is not gated (they ride on the project DTO; absent fields deserialise as empty). No environment enablement is part of this work.

Domain model (slice A)

  • Project.ScreeningInclusionKeywords and Project.ScreeningExclusionKeywords: IReadOnlyList<string> backed by private List<string>? fields, mapped with MapField(...).SetElementName(...).SetIgnoreIfNull(true) in ProjectRepository.RegisterProjectAndJobMappings, so existing documents need no migration and store no element until first set. Included in the summary projection so every project read carries them.
  • Single mutator Project.SetScreeningKeywords(IEnumerable<string> inclusion, IEnumerable<string> exclusion) applying ScreeningKeywordRules:
  • trim, collapse internal whitespace to one space, drop empties;
  • de-duplicate case-insensitively within a list (first occurrence wins, original casing kept);
  • limits: 100 characters per term, 200 terms per list (ArgumentException → 400);
  • a term that appears in both lists (case-insensitive) is rejected (400); the reviewer-facing semantics of such a term are undefined, and overlapping phrases are handled at render time instead.
  • Existing Project.Keywords (project metadata tags, JSON-Patch path /keywords, anonymous GET api/projects/keywords) is unrelated and untouched; the new names are deliberately explicit.

API (slice A)

  • PUT api/projects/{projectId:guid}/screening-keywords, body ScreeningKeywordsUpdateDto { inclusionKeywords: string[], exclusionKeywords: string[] }, [Authorize(ProjectAuthorization.ProjectDesignPolicy)] (the existing project-administrator gate, Administrator group by default), [FeatureGated(Flag.ScreeningKeywordHighlighting)]. Returns ScreeningKeywordsDto { projectId, inclusionKeywords, exclusionKeywords } with the normalised lists.
  • Read: both lists added to ProjectDbBaseDto, so the generated web IProject gains screeningInclusionKeywords / screeningExclusionKeywords and the stage review page's existing selectCurrentProject signal carries them with no extra request.
  • Modelled on the category-guidance endpoint (commit ffadc82ec).

Settings UI (slice A)

Screening Settings page (project/:projectId/admin/screening-settings, already guarded by projectDesignGuardFn): a new "Keyword highlighting" section under the criteria block, flag-gated, with two app-chip-input lists (inclusion, exclusion), save/discard, a dirty guard integrated into the page's existing canDeactivate, and explanatory copy: highlights are visual only; whole-word matching; trailing * for prefix matching; highlights need a PDF with extractable text. Save goes through a new projectDetailActions.updateScreeningKeywords effect → generated client → reducer merges the returned lists into the project entity.

Matching model (slice B, pure TypeScript, unit-tested)

  • Input: the page's text-content items (TextLayer.textContentItemsStr), joined in order; an item with hasEOL contributes a newline. Positions map back to the item index and offset.
  • Case-insensitive (NFKC normalisation + lower-casing); no diacritic folding (follow-up).
  • Whitespace inside a phrase matches any run of whitespace, including line breaks.
  • Whole-word by default: a match must not be preceded or followed by a Unicode letter or digit. A trailing * on a term removes the trailing boundary (prefix match). No other operator syntax; all other characters are literal.
  • Overlaps and duplicates: matches from both lists are painted into a per-character mask; the mask is then cut into segments classified inclusion, exclusion or both. Nested, overlapping and repeated terms therefore merge without double-wrapping.
  • Cap: if a page yields more than 5,000 matches the highlighter stops and reports the cap (defensive).

Rendering (slice B)

  • After each successful page render in StudyPdfViewerComponent._renderPage, when a keyword config is present: page.getTextContent() → new TextLayer({ textContentSource, container, viewport }).render() into a <div class="textLayer"> that is a sibling of the canvas inside a position: relative wrapper sized to the canvas CSS width, with --scale-factor set to the render scale. The text layer is cancelled/cleared on the same paths that cancel the canvas render (request token, _settleActiveRender, _destroyDocument), and a cancelled or superseded render never paints.
  • Highlights wrap matched ranges in <mark class="syrf-keyword syrf-keyword--inclusion|--exclusion|--both"> inside the text-layer spans. Text-layer text is transparent; marks use mix-blend-mode: multiply so the PDF glyphs underneath stay readable. Elements are created through the container's ownerDocument so Dockview panel windows (live DOM relocation) work.
  • Accessibility: colour is never the only cue. Inclusion marks carry a solid 2px bottom border, exclusion marks a double bottom border, both shows both; each mark has aria-label="Inclusion keyword" / "Exclusion keyword". Colours are theme tokens (--syrf-keyword-inclusion-*, --syrf-keyword-exclusion-*) defined in both the light and .global-dark-theme blocks of syrf-theme.scss; the PDF page stays white in both themes, so the tokens are tuned for a white page.
  • Status line under the page navigation: "Keywords: N inclusion, M exclusion on this page" plus a legend swatch; when the page has no text items: "This page has no extractable text, so keyword highlights cannot be shown." SyRF does not perform OCR and the copy does not promise it.
  • Live configuration: the viewer takes keywordConfig = input<ScreeningKeywordConfig | null>(null); a change re-runs the highlighter on the existing text layer without reloading the document or altering _applyState, revision, retry or heartbeat semantics. The panel's _pdfIdentity key is unchanged, so a keyword change never remounts the viewer.
  • Study switches: the viewer is recreated per study by the host; all highlight state is input-derived.
  • Manual search and ordinary interaction: no key handling is added; the text layer makes browser find-in-page and text selection work on the rendered page, which they did not before.
  • Popup viewer (/pdf-viewer): unchanged in slice B; it shows no highlights until the protocol follow-up extends StudyPdfViewerStudy with an optional field (rolling-deploy tolerant, like directUrl).

Decisions made here that Chris may want to revisit

  1. Whole-word matching with trailing * as the only wildcard (alternative: substring matching).
  2. Rejecting a term present in both lists at save time; overlapping phrases render as both.
  3. Keyword lists live on the Screening Settings page under the project-design permission, not on General Settings and not per stage.
  4. Highlight colours and the non-colour cues (solid vs double underline).
  5. Popup-viewer parity deferred to a follow-up.

Testing

  • .NET: ScreeningKeywordRules unit tests; BsonClassMapTests round-trip and absent-element tests; controller authorization/route attribute test; RuntimeFeatureFlagCatalogTests and ProjectAuthorizationActivityParityTests remain green.
  • Web: matcher unit tests (case, phrase whitespace, word boundaries, prefix wildcard, overlaps, duplicates, cap); highlighter DOM tests against a fake text layer; viewer spec additions (render → text layer → marks; cancelled render paints nothing; config change re-highlights without reload; no-text page notice); panel and stage-review wiring specs; settings component spec; effects/reducer specs; guard specs (syrf-theme.spec.ts, no-hardcoded-help-urls.spec.ts).
  • Browser check on a preview environment with the seeded "Ready for Annotation" project (fixtures under assets/seed-pdf-fixtures/seed/ contain selectable text).