deduplication - Event Deduplication with Canonical Selection and Reputation Scoring
Deduplicates events using canonical selection, reputation scoring, and hash-based grouping for multi-source data aggregation
Tags
Updated: 2026-02-11Capabilities
Typical Inputs
Typical Outputs
What this skill does
- generate deduplication keys
- score source reputation
- select canonical versions
- group similar content
- track reduction metrics
Inputs
- event data
- source results
- article metadata
- domain information
- URLs
Outputs
- deduplicated items
- reduction percentage
- canonical selections
- duplicate groups
Requirements
- TypeScript/JavaScript environment
- event data sources
- article metadata