Menu

Individual skill

smart-data-collection

0upvotes

智能数据采集技能，用于从图片或文档（PDF、Word、Excel）中提取结构化数据，基于知识网络完成字段映射，生成SQL并写入数据库。当用户提到"数据采集"、"从文档提取数据"、"图片转数据"、"数据导入"、"文档数据入库"、"批量数据提取"或需要从非结构化文件中提取结构化数据并存储时，自动使用此技能。

configs/microservice/bff-service/configs/agent-skills/ontology/smart-data-collection/SKILL.md

Read pinned source

Source commit 3f908c703fd75033217ed549f1998563d13a1dc0

What was checked

Matches the observed repository HEAD
Yes
Scanner rules are current
Yes
Repository scan complete
No
Skill scan complete
No
Observed behavior
Network or broad shell
Last checked
Nov 9, 2026

These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.

Install this skill

Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.

An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.

Repository context

Repository stars (not skill ratings)
2,386
Repository last updated
Nov 6, 2026
License
Apache-2.0

Observed agent integration: No specific integration evidence. Standard SKILL.md installation does not imply verified compatibility.

Files and excerpt

File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.

Browse files
Read excerpt

# Smart Data Collection（数据采集 / 写入）

本 skill 定义 **固定顺序** 的数据写入工具链，完全由 `ontology` **native 子命令**（`bkn` / `dataview` / `ds`）实现；
所有 CLI 执行均委托 [ontology-core](../ontology-core/SKILL.md)。

## 安全红线（IRON RULE）

1. **严禁直连数据库**：不得使用 `mysql` / `mysqldump` / `pymysql` / `sqlalchemy+pymysql` / 直接 JDBC 连接等方式向 `data_source` 写入数据。
   - `ontology ds get` 返回的 `bin_data.password` 是 **平台加密串**，不是明文密码；尝试解密或直连均视为违规。
   - 直连绕过平台审计、权限、字段映射、索引刷新与事务一致性，会导致 BKN 索引与底表脱节。
2. **唯一允许的写入入口**：`ontology ds import-csv <datasource_id> --table-name <table> --file <csv> [...]`
   - 由 CLI 调用后端 `POST /api/automation/v1/data-flow/flow`，由平台 data-flow 服务统一执行写入与回执。
3. **`import-csv` 仅支持 INSERT，不支持 UPDATE/UPSERT/REPLACE**：
   - 每行执行一次 `INSERT`；若主键已存在，MySQL 直接报 `Error 1062 (23000): Duplicate entry '<pk>' for key '<table>.PRIMARY'`，整批失败、`summary.failed=1`、`rows_written=0`。