LogoClawIndex
CasesSkillsAbout
LogoClawIndex

deduplication - Event Deduplication with Canonical Selection

Removes duplicate events using hash-based grouping and selects authoritative versions from multiple sources

Tags

Updated: 2026-02-11
data-processingdeduplicationaggregationcanonical-selection

Capabilities

generate deduplication keysgroup by semantic similarityscore source reputationselect canonical versions

Typical Inputs

multiple data sourcesarticles with IDscontent metadata

Typical Outputs

deduplicated items listreduction percentage reportsource attribution data

What this skill does

  • generate deduplication keys
  • group by semantic similarity
  • score source reputation
  • select canonical versions
  • calculate reduction metrics

Inputs

  • multiple data sources
  • articles with IDs
  • content metadata
  • URLs
  • source reputation data

Outputs

  • deduplicated items list
  • reduction percentage report
  • source attribution data
  • duplicate groups count

Requirements

  • TypeScript/JavaScript environment
  • crypto module for hashing
  • data with unique identifiers

Source

  • Spec: SKILL.md

ClawIndex

OpenClaw Skills & Use Case Index

ClawIndex is an ecosystem-driven index of OpenClaw skills and real-world use cases.

Index

Skills·
Cases

Meta

About·
Disclaimer·
Email·
GitHub
© 2026 ClawIndex All Rights Reserved.