Data Engineering Bundle | Prompeteer.ai

Pipeline architecture, dbt models, schema design, lineage docs, and data-quality skills for data engineers.

Browse all skill bundles

Included Skills (65)

Dbt Sf To Bq Translator — The skill translates Snowflake dbt SQL models into standardized BigQuery SQL, enabling seamless migration of data pipelines. It compiles SQL, masks Jinja macro placeholders, applies BigQuery Translation Service workflows, transforms AST-based configurations, casts types explicitly, standardizes JSON extraction, and removes duplicate records. Professionals migrating Snowflake dbt pipelines to Google Cloud BigQuery benefit from accurate, efficient conversion, while the tool is not intended for generic BigQuery queries or other SQL migrations.
Apache Spark — Apache Spark is a distributed engine that processes batch, SQL, and streaming data, accessible through Python via PySpark. It enables users to perform large‑scale computations, build ETL pipelines, analyze data at scale, read Parquet files from S3, and stream from Kafka, making it essential for data engineers and analysts handling big data workloads.
Great Expectations — Great Expectations (GX Core) is a Python framework that enables data professionals to define precise expectations for datasets, such as null counts, value ranges, uniqueness, and allowed values. It validates pandas DataFrames or SQL tables against these expectations, integrates checkpoints into data pipelines, and produces Data Docs that clearly display which checks passed or failed. Analysts, data engineers, and data scientists benefit from reliable, documented data quality assurance.
Data Lineage Summarizer — This tool generates concise Markdown reports from Google Cloud Data Lineage graphs, helping data engineers trace provenance and troubleshoot complex data quality issues.
Data Enrichment Optimizer — This skill helps RevOps and data engineering teams maximize data quality and minimize costs by optimizing provider selection and waterfall enrichment sequences.
Bigquery Observability — The Bigquery Observability skill provides data engineers with proven best practices, tool selection guidance, and high‑performance SQL syntax for querying BigQuery telemetry across INFORMATION_SCHEMA, Cloud Monitoring, and the REST API. It enables rapid diagnosis of single‑job bottlenecks, slot contention, job concurrency, queue latency, reservation capacity, utilization, autoscaling saturation, and audit compute and storage usage. The skill is tailored for known telemetry needs and performance analysis, not for root‑cause diagnosis or business‑logic optimization.
Bigquery Optimization — The Bigquery Optimization skill refines BigQuery environments through structured workflows that include capacity planning, edition migration, and reservation rightsizing. It streamlines storage management with partitioning, clustering, lifecycle policies, and billing model adjustments while generating DDL and optimizing individual SQL queries. Data engineers and analysts benefit from reduced costs, improved performance, and a more efficient data architecture.
Dbt Transformation Architect — This skill provides production-ready dbt patterns and best practices to help data engineers build scalable, well-documented, and efficient transformation pipelines.
Soda Data Validator — Assists developers in writing YAML-based data quality checks using Soda, ensuring data freshness, completeness, and validity for reliable data pipelines.
Cloud Monitoring Metric Selection — The skill retrieves and queries Cloud Monitoring metric descriptors for Google Cloud services such as Compute Engine, Spanner, BigQuery, Cloud Run, Cloud SQL, Pub/Sub, and Cloud Storage. It identifies relevant metrics, including their names, types, and value schemas, to support discovery and selection tasks. Engineers and DevOps teams use it to quickly locate the metrics needed for monitoring and alerting.
BigQuery Data Analyzer — Analyze large datasets using Google BigQuery, enabling data professionals to run SQL queries and build machine learning models.
Embedded Analytics Database — This skill helps data professionals use DuckDB, an embedded analytical database, for local data exploration and ETL pipelines.
Real-Time Analytics Pipeline — This skill helps AI agents build real-time analytics pipelines, including event ingestion, ClickHouse storage, and dashboard wiring for live metrics.
Kafka Event Streaming — Facilitates building event-driven systems using Apache Kafka for users needing message streaming and real-time data processing.
CSV Data Processor — This skill parses and generates CSV files, assisting developers with data import/export, ETL pipelines, and report generation tasks.
Data Warehouse Analyst — This tool helps data analysts plan, validate, and review read-only warehouse queries to ensure accurate, privacy-compliant, and reproducible business insights.
Data Migration — The Data Migration skill analyzes source and target schemas, maps fields, and processes data in batches while validating outcomes to ensure accurate transfers. It transforms schemas for seamless integration between databases, providing reliable, comprehensive support that database administrators and data engineers rely on to move data efficiently and accurately.
DynamoDB Skill Builder — This skill helps developers design and build efficient NoSQL databases using Amazon DynamoDB for scalable applications.
JSON Processing Expert — This skill provides expert guidance on using jq to query, filter, and transform complex JSON data within shell pipelines for developers and engineers.
SQL Table Reconciler — This tool compares SQL Server tables across instances to validate data migrations, detect schema drift, and generate detailed row-level reconciliation reports for engineers.
API Integration Architect — Designs robust event-driven architectures, webhook systems, and data pipelines to help developers seamlessly connect and orchestrate complex service-to-service communication flows.
Analytics Quality Framework — This skill helps data engineers and analysts establish robust testing, automated monitoring, and incident response protocols for analytics models and dashboards.
Postgres Egress Optimizer — This tool helps developers identify and refactor inefficient query patterns to reduce excessive data transfer costs from their Neon Postgres database.
NeMo Recipe Architect — This tool assists developers in creating, configuring, and validating NeMo AutoModel training and evaluation recipes through structured YAML and builder management.
Google Cloud Storage Basics — Google Cloud Storage Basics teaches users how to store, retrieve, and manage data as objects in GCS buckets, covering bucket configuration, data transfer, access control, and lifecycle management. Developers and administrators use the skill to interact with Cloud Storage through gsutil, APIs, or Terraform, while remaining focused on object storage rather than block storage or databases.
Dataverse Solution Architect — This tool helps developers design and implement robust Dataverse SDK solutions by providing architectural guidance, best practices, and production-ready Python code.
SQL Performance Optimizer — This tool helps database administrators and developers improve query efficiency by analyzing execution plans and recommending optimal indexing strategies for major SQL database systems.
Time Series Anomaly Detector — This tool enables fine-tuning and inference of diffusion-based models to identify anomalies in multivariate time-series data for data scientists and engineers.
Prisma Upgrade V7 — The Prisma Upgrade V7 skill delivers a detailed migration guide that maps every breaking change from Prisma ORM v6 to v7 and prescribes the necessary code adjustments. It helps developers and teams resolve errors and transition projects smoothly, ensuring uninterrupted operation of applications that rely on Prisma’s data access layer. This skill is indispensable for anyone maintaining or expanding Prisma‑based applications.
EF Core Diagram Generator — This skill generates D2 database diagrams from Entity Framework Core models to help developers visualize table relationships, constraints, and schema structures effectively.
Fluentd Log Management — Configure Fluentd for collecting, filtering, and routing logs across distributed systems, assisting developers and DevOps engineers with log aggregation.
Azure Table Integrator — This tool assists Java developers in building scalable table storage applications compatible with both Azure Table Storage and Cosmos DB Table API.
Structured Output — Structured Output directs large language models to generate validated JSON, ensuring machine‑readable data integrity. Developers and data engineers use it to extract and validate structured information from OpenAI, Anthropic, or Claude APIs, integrating seamlessly with Zod schemas, Pydantic models, and the Instructor library. Its retry strategies for malformed responses guarantee robust data extraction.
Cosmos DB Architect — This tool assists developers in building secure, production-ready Azure Cosmos DB NoSQL services using clean code and test-driven development principles.
Performance Optimization Specialist — This tool identifies and resolves performance bottlenecks in code, databases, and APIs by providing measurable, data-driven improvements for developers seeking faster applications.
Test Coverage Optimizer — This tool identifies untested code paths and generates targeted unit tests to help developers systematically increase their project's overall test coverage metrics.
Temporary Postgres Provisioner — This tool enables developers to instantly provision temporary, account-free Postgres databases for rapid prototyping, testing, and ephemeral development environment needs.
Weather Data Retriever — This tool enables developers to efficiently fetch specific numerical weather prediction data from public archives using optimized byte-range requests and verified caching strategies.
Azure Table Storage — This tool enables developers to manage NoSQL key-value data and perform entity operations within Azure Storage Tables or Cosmos DB Table API.
Python Visualization Assistant — This skill provides expert guidance on creating publication-quality static, animated, and interactive data visualizations using the Matplotlib library for Python developers.
Redis CLI Assistant — This tool provides comprehensive command-line interface guidance to help developers efficiently query, debug, and manage their Redis databases through interactive terminal commands.
Database Migration Specialist — This skill assists developers in executing seamless database schema migrations, data transformations, and zero-downtime deployments across various popular ORM frameworks.
TypeScript Idiomatic Guide — This skill provides developers with best practices and idiomatic patterns to write cleaner, more efficient, and maintainable TypeScript and JavaScript code.
API Interface Architect — Assists developers in designing stable, well-documented, and robust interfaces for APIs, modules, and system boundaries to ensure long-term maintainability and usability.
Parallel Batch Processor — This tool enables developers to execute high-performance, parallel file operations and bulk data transformations across large directories with integrated progress tracking and error handling.
Ms Access — The skill designs, builds, and maintains Microsoft Access databases, creating tables, relationships, queries, forms, reports, and VBA automation. It enables seamless migration to SQL Server, Dataverse, or open‑source databases and supports reading .accdb/.mdb files from scripts. Clients requiring robust Access solutions—from database creation to web‑app conversion—benefit from this comprehensive expertise.
Cosmos DB Manager — Manage Azure Cosmos DB resources, enabling developers to build globally distributed, multi-model database applications efficiently.
SQL Query Optimizer — Analyzes and optimizes SQL queries for improved performance, assisting developers and database administrators in enhancing database efficiency.
Mikro Orm — MikroORM provides a TypeScript ORM for Node.js that implements Data Mapper, Unit of Work, and Identity Map patterns, enabling developers to define decorator-based entities and automatically track changes across PostgreSQL, MySQL, MariaDB, SQLite, and MongoDB. It supports query‑builder queries, migration generation and execution, and database seeding, making it ideal for building a domain‑driven, TypeScript‑friendly data layer.
Axiom Expert Guide — Provides expert guidance for developers using Axiom, the serverless log management and analytics platform, to ingest, query, and visualize data.
MotherDuck SQL Assistant — Provides expert guidance for MotherDuck, helping developers perform SQL analytics on cloud data and build hybrid local-cloud data pipelines.
Apache Arrow Guide — Provides expert assistance for Apache Arrow, enabling developers to utilize its high-performance columnar format for data interchange and processing.
JSON Data Shuffler — This tool assists data engineers by safely randomizing repetitive JSON objects while ensuring structural integrity and schema consistency throughout the entire process.
Oracle Postgres Migrator — This tool automates the migration of .NET data access layers from Oracle to PostgreSQL by refactoring code, updating dependencies, and mapping database types.
Mongoose — Mongoose supplies a structured ODM for MongoDB in Node.js, letting developers define schemas that enforce field types, validation, and defaults. It offers typed queries, middleware hooks, reference population, and transaction support, streamlining data integrity and workflow. Backend engineers and full‑stack developers benefit from robust, type‑safe MongoDB interactions and a smooth migration to Mongoose 9.
Data Version Control — DVC tracks large datasets and ML models, builds reproducible pipelines, and enables experiment tracking for data scientists and ML engineers.
Dataverse Python Architect — This tool helps developers generate robust, production-ready Python code for Dataverse by implementing professional error handling, connection management, and performance optimization best practices.
Orama — Orama provides full‑text, vector, and hybrid search directly in JavaScript and TypeScript, enabling developers to build robust on‑premise search solutions without external services. It allows teams creating document sites, client‑side applications, or typo‑tolerant faceted search to integrate embeddings, filters, and persistent indexing for precise, efficient retrieval.
Data Science Expert — This skill provides advanced analytics, machine learning, and statistical modeling support to help data professionals execute complex data workflows and generate actionable business insights.
Supabase Management Assistant — Automate Supabase database operations, schema management, and project administration for developers using the Rube MCP toolkit to streamline backend workflows efficiently.
Database Architecture Specialist — This expert architect assists developers in designing scalable data layers, selecting optimal technologies, and planning robust schema migrations for high-performance applications.
Cloud Storage Specialist — This expert assistant helps developers implement secure, high-performance file upload systems using presigned URLs, multipart streaming, and robust server-side validation techniques.
MongoDB Schema Architect — This skill provides expert guidance on MongoDB data modeling patterns and anti-patterns to help developers build performant, scalable, and efficient database schemas.
Alembic — Manage database migrations for SQLAlchemy with Alembic. Use when a user asks to version database schemas, autogenerate migration scripts from models, run or roll back migrations, set up async SQLAlchemy migrations, or handle schema changes safely in production.
N8n Workflow Orchestrator — This router streamlines n8n MCP workflow development, testing, and debugging by directing tasks to specialized guidance for developers and automation engineers.