Skills tagged: data-processing

• polars - Fast in-memory DataFrame library based on Apache Arrow

  polarsdataframeetlpython

  ★ 0 · Updated 2026-09-22

  Provides data manipulation capabilities, lazy query optimization, file I/O operations, and pandas migration patterns using Polars.

  ⚙ Query and manipulate tabular data⚙ Perform group aggregations⚙ Apply window functions
• python-executor - Execute Python code in a safe sandboxed environment

  pythonsandboxcode-executionweb-scraping

  ★ 0 · Updated 2026-09-16

  Executes Python code in a sandboxed environment via inference.sh with pre-installed libraries for data processing, web scraping, and media generation.

  ⚙ Execute Python code⚙ Scrape web pages⚙ Process and visualize data
• polars - Process and manipulate tabular data efficiently with Polars.

  polarsdataframepythondata-processing

  ★ 6 · Updated 2026-09-09

  Polars is an in-memory DataFrame library featuring expression-based transformation, lazy query optimization, and multi-format data I/O.

  ⚙ Filter and transform DataFrames⚙ Execute lazy query pipelines⚙ Group and aggregate tabular data
• polars - Fast DataFrame Processing with Polars

  dataframedata-processingetlpandas-alternative

  ★ 18 · Updated 2026-06-30

  In-memory DataFrame library built on Apache Arrow with lazy evaluation and parallel execution for efficient data processing of 1-100GB datasets.

  ⚙ read CSV/Parquet/JSON data files⚙ write data to multiple formats⚙ filter DataFrame rows
• astrophysics-data-guide - Process astronomical data with Astropy

  astronomyastrophysicsdata-processingFITS

  ★ 3,789 · Updated 2026-06-30

  Process FITS files, coordinates, photometry, spectra, and sky survey data

  ⚙ read FITS files⚙ inspect FITS structure⚙ read FITS image
• databricks-connect - Databricks Connect Guide for PySpark Development

  databrickspysparkbig-datadata-processing

  ★ 0 · Updated 2026-03-09

  Guide for writing code with Databricks Connect and debugging connection issues

  ⚙ create DatabricksSession⚙ read data from tables⚙ write data to tables
• architecture-paradigm-pipeline - Pipeline Architecture Pattern for Data Transformation Workflows

  architecturepipelinepipes-filtersETL

  ★ 18 · Updated 2026-02-18

  Design data pipelines or transformation workflows using pipes and filters paradigm

  ⚙ design pipeline architecture⚙ implement filters⚙ connect with pipes
• jq-yq - JSON and YAML Data Processing with jq and yq Tools

  data-processingjsonyamlcommand-line

  ★ 3 · Updated 2026-02-17

  Manipulate and transform JSON and YAML data using jq and yq command-line tools

  ⚙ process JSON data⚙ process YAML data⚙ convert YAML to JSON
• jq-yq - Command-line JSON and YAML data manipulation with jq and yq

  jsonyamlclidata-processing

  ★ 18 · Updated 2026-02-17

  Manipulate JSON and YAML data using jq and yq command-line tools

  ⚙ parse JSON⚙ transform YAML⚙ extract JSON fields
• jq-yq - Command-line JSON and YAML Processing with jq and yq Tools

  jsonyamldata-processingcommand-line

  ★ 602 · Updated 2026-02-17

  Process and manipulate JSON and YAML data using jq and yq command-line tools

  ⚙ parse JSON data⚙ transform YAML data⚙ extract fields from JSON
• dfdl_ref - DataFusion and DeltaLake Operations Reference

  data-processingquery-enginestorage-layeroperations

  ★ 0 · Updated 2026-02-13

  Provides operational guidance for DataFusion query engine and DeltaLake storage layer integration

  ⚙ read reference files⚙ grep code patterns⚙ glob file paths
• yq - Efficient YAML field extraction tool for targeted data retrieval

  yamlqueryextractiondata-processing

  ★ 20 · Updated 2026-02-12

  Extracts specific fields from YAML files without reading entire file contents

  ⚙ query yaml files⚙ extract fields⚙ filter data
• yq - Efficient YAML Field Extraction Tool

  yamlextractionquerydata-processing

  ★ 18 · Updated 2026-02-12

  Extract specific fields from YAML files using query expressions without reading entire files.

  ⚙ extract fields from YAML⚙ query nested structures⚙ filter YAML data
• write-script-duckdb - DuckDB Query Script Writer for Data Operations

  DuckDBSQLdata-processingdatabase

  ★ 6 · Updated 2026-02-12

  Writes DuckDB queries for data operations and integrations

  ⚙ write DuckDB queries⚙ integrate Ducklake operations⚙ connect external databases
• jq - jq JSON Processor for Command Line Data Manipulation

  JSONcommand-linedata-processingfiltering

  ★ 602 · Updated 2026-02-12

  Process JSON data through filtering, transformation, and manipulation in command line

  ⚙ filter JSON fields⚙ transform JSON structures⚙ manipulate JSON data
• csv - CSV File Processing and Analysis

  csvdata-processingbashfile-management

  ★ 0 · Updated 2026-02-12

  Process and analyze CSV data using bash command line tools

  ⚙ get CSV statistics⚙ filter CSV rows⚙ select CSV columns
• deduplication - Event Deduplication with Canonical Selection and Reputation Scoring

  data-processingdeduplicationmulti-sourceaggregation

  ★ 18 · Updated 2026-02-11

  Deduplicates events using canonical selection, reputation scoring, and hash-based grouping for multi-source data aggregation

  ⚙ generate deduplication keys⚙ score source reputation⚙ select canonical versions
• deduplication - Event Deduplication with Canonical Selection and Reputation Scoring

  data-processingdeduplicationaggregationcanonical-selection

  ★ 211 · Updated 2026-02-11

  Deduplicate events from multiple sources with canonical selection and reputation scoring

  ⚙ deduplicate events by ID⚙ deduplicate articles by content⚙ generate deduplication keys
• deduplication - Event Deduplication with Canonical Selection and Reputation Scoring

  data-processingduplicatesaggregationmulti-source

  ★ 602 · Updated 2026-02-11

  Remove duplicate events from multiple sources using ID-based and content-based deduplication with canonical selection

  ⚙ generate deduplication key⚙ calculate reputation score⚙ group similar content
• csv - CSV data processing with bash tools

  data-processingcsvbashutilities

  ★ 0 · Updated 2026-02-11

  Process CSV files using standard bash tools like awk, cut, sort, and grep

  ⚙ analyze CSV statistics⚙ filter CSV rows⚙ select CSV columns

Scroll to load more