增加flask作为后台，查看products.db数据库表的功能
This commit is contained in:
Binary file not shown.
|
Before Width: | Height: | Size: 281 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 166 KiB |
@@ -1,135 +0,0 @@
|
||||
# 系统合并总结报告
|
||||
|
||||
## 合并概述
|
||||
|
||||
成功将 `integrated_scraper.py` 和 `product_ai_analysis.py` 合并为一个统一的系统，提供完整的产品抓取和AI分析功能。
|
||||
|
||||
## 功能特性对比
|
||||
|
||||
| 功能 | integrated_scraper.py | product_ai_analysis.py | 合并后系统 |
|
||||
|------|----------------------|----------------------|-------------|
|
||||
| 产品抓取 | ✅ | ❌ | ✅ |
|
||||
| AI分析 | ❌ | ✅ | ✅ |
|
||||
| 数据库操作 | ✅ | ✅ | ✅ |
|
||||
| 配置管理 | ❌ | ❌ | ✅ |
|
||||
| 统一命令行 | ❌ | ❌ | ✅ |
|
||||
| 错误处理 | 基础 | 基础 | 增强 |
|
||||
| 日志记录 | ✅ | ✅ | ✅ |
|
||||
| 进度显示 | ✅ | ❌ | ✅ |
|
||||
|
||||
## 新文件结构
|
||||
|
||||
```
|
||||
product/
|
||||
├── integrated_product_system.py # 核心系统（合并后）
|
||||
├── run_system.py # 命令行界面
|
||||
├── config.py # 配置文件
|
||||
├── README.md # 使用说明
|
||||
└── MERGE_SUMMARY.md # 合并总结
|
||||
```
|
||||
|
||||
## 核心改进
|
||||
|
||||
### 1. 统一接口
|
||||
- 单一入口点处理抓取和分析
|
||||
- 支持多种运行模式（抓取/分析/完整）
|
||||
- 统一的配置管理和错误处理
|
||||
|
||||
### 2. 配置管理
|
||||
- 集中化的配置系统
|
||||
- 支持命令行参数覆盖
|
||||
- 灵活的数据库和API配置
|
||||
|
||||
### 3. 用户体验
|
||||
- 简化的命令行界面
|
||||
- 详细的进度显示和日志记录
|
||||
- 增强的错误处理和恢复机制
|
||||
|
||||
### 4. 代码质量
|
||||
- 模块化设计，职责分离
|
||||
- 增强的错误处理和日志记录
|
||||
- 完整的文档和示例
|
||||
|
||||
## 使用方式对比
|
||||
|
||||
### 原系统使用方式
|
||||
```bash
|
||||
# 抓取产品
|
||||
python integrated_scraper.py --limit 100
|
||||
|
||||
# 分析产品（需要单独运行）
|
||||
python product_ai_analysis.py --max-products 50
|
||||
```
|
||||
|
||||
### 合并后系统使用方式
|
||||
```bash
|
||||
# 完整工作流程（抓取+分析）
|
||||
python run_system.py --mode full --limit 100 --max-products 50
|
||||
|
||||
# 仅抓取
|
||||
python run_system.py --mode scraping --limit 100
|
||||
|
||||
# 仅分析
|
||||
python run_system.py --mode analysis --max-products 50
|
||||
```
|
||||
|
||||
## 功能增强
|
||||
|
||||
### 新增功能
|
||||
- ✅ 统一的工作流程管理
|
||||
- ✅ 灵活的运行模式选择
|
||||
- ✅ 增强的配置系统
|
||||
- ✅ 改进的命令行界面
|
||||
- ✅ 更好的错误处理
|
||||
|
||||
### 保留功能
|
||||
- ✅ 产品抓取功能（完整保留）
|
||||
- ✅ AI分析功能（完整保留）
|
||||
- ✅ 数据库结构（完全兼容）
|
||||
- ✅ 日志记录（增强版）
|
||||
- ✅ 进度显示（改进版）
|
||||
|
||||
## 向后兼容性
|
||||
|
||||
- ✅ 数据库结构完全兼容
|
||||
- ✅ 保留所有原有功能
|
||||
- ✅ 支持原有命令行参数
|
||||
- ✅ 兼容现有数据文件
|
||||
|
||||
## 测试验证
|
||||
|
||||
✅ 系统导入测试通过
|
||||
✅ 数据库初始化测试通过
|
||||
✅ 命令行界面测试通过
|
||||
✅ 配置系统测试通过
|
||||
✅ 异步调用修复测试通过
|
||||
|
||||
所有测试均成功完成，系统功能正常。
|
||||
|
||||
## 重要修复记录
|
||||
|
||||
### 异步调用问题修复 (v1.0.1)
|
||||
**问题**: `asyncio.run() cannot be called from a running event loop`
|
||||
**原因**: 在已有事件循环中嵌套调用 `asyncio.run()`
|
||||
**解决方案**:
|
||||
- 重构 `run_full_workflow` 方法为异步版本 `run_full_workflow_async`
|
||||
- 新增同步入口方法，使用新事件循环执行异步函数
|
||||
- 主函数直接调用异步版本，避免嵌套事件循环
|
||||
**状态**: ✅ 已修复并测试通过
|
||||
|
||||
## 迁移指南
|
||||
|
||||
### 从原系统迁移
|
||||
1. 备份现有数据库文件
|
||||
2. 使用新的命令行界面
|
||||
3. 根据需要调整配置文件
|
||||
4. 验证数据完整性
|
||||
|
||||
### 配置迁移
|
||||
- 原有参数大部分兼容
|
||||
- 新增配置选项可选使用
|
||||
- 支持环境变量和配置文件
|
||||
|
||||
## 总结
|
||||
|
||||
合并后的系统提供了更简洁、更强大、更易用的产品抓取和AI分析功能，同时保持了与原有系统的完全兼容性。通过统一的接口和增强的配置管理，大大简化了使用流程，提高了开发效率。
|
||||
File diff suppressed because it is too large
Load Diff
@@ -102,13 +102,13 @@ class IntegratedProductSystem:
|
||||
''')
|
||||
|
||||
# 创建分析结果表（来自product_ai_analysis.py）
|
||||
# 移除了product_intro字段，避免与ai_response内容重复
|
||||
cursor.execute('''
|
||||
CREATE TABLE IF NOT EXISTS product_analysis (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
original_id INTEGER,
|
||||
original_name TEXT,
|
||||
product_name TEXT,
|
||||
product_intro TEXT,
|
||||
development_difficulty TEXT,
|
||||
difficulty_score INTEGER,
|
||||
ai_response TEXT,
|
||||
@@ -367,17 +367,16 @@ class IntegratedProductSystem:
|
||||
|
||||
def save_analysis_result(self, conn: sqlite3.Connection,
|
||||
original_id: int, original_name: str,
|
||||
product_name: str, product_intro: str,
|
||||
difficulty: str, ai_response: str):
|
||||
product_name: str, difficulty: str, ai_response: str):
|
||||
"""保存分析结果到数据库"""
|
||||
try:
|
||||
cursor = conn.cursor()
|
||||
|
||||
cursor.execute("""
|
||||
INSERT INTO product_analysis
|
||||
(original_id, original_name, product_name, product_intro, development_difficulty, ai_response)
|
||||
VALUES (?, ?, ?, ?, ?, ?)
|
||||
""", (original_id, original_name, product_name, product_intro, difficulty, ai_response))
|
||||
(original_id, original_name, product_name, development_difficulty, ai_response)
|
||||
VALUES (?, ?, ?, ?, ?)
|
||||
""", (original_id, original_name, product_name, difficulty, ai_response))
|
||||
|
||||
conn.commit()
|
||||
logger.success(f"保存分析结果成功: {product_name}")
|
||||
@@ -438,9 +437,9 @@ class IntegratedProductSystem:
|
||||
# 解析响应
|
||||
product_name, product_intro, difficulty = self.parse_ai_response(ai_response)
|
||||
|
||||
# 保存结果
|
||||
# 保存结果（不再保存product_intro，避免与ai_response重复）
|
||||
self.save_analysis_result(conn, original_id, name,
|
||||
product_name, product_intro, difficulty, ai_response)
|
||||
product_name, difficulty, ai_response)
|
||||
success_count += 1
|
||||
|
||||
# 显示完成状态
|
||||
|
||||
@@ -1,10 +0,0 @@
|
||||
2025-11-28 22:29:36.321 | INFO | __main__:__init__:109 - 初始化SQLite数据库查看器
|
||||
2025-11-28 22:29:36.322 | INFO | __main__:init_ui:116 - 设置主窗口界面
|
||||
2025-11-28 22:29:36.324 | INFO | __main__:create_top_buttons:146 - 创建顶部按钮
|
||||
2025-11-28 22:29:36.330 | INFO | __main__:create_filter_section:169 - 创建筛选控件区域
|
||||
2025-11-28 22:29:36.347 | INFO | __main__:create_splitter:214 - 创建分割器界面
|
||||
2025-11-28 22:29:36.361 | INFO | __main__:create_status_bar:261 - 创建状态栏
|
||||
2025-11-28 22:29:36.362 | INFO | __main__:create_menubar:268 - 创建菜单栏
|
||||
2025-11-28 22:29:36.389 | INFO | __main__:init_ui:142 - 界面初始化完成
|
||||
2025-11-28 22:29:36.572 | INFO | __main__:main:669 - 应用程序启动完成
|
||||
2025-11-28 22:29:40.545 | INFO | __main__:closeEvent:647 - 关闭应用程序
|
||||
@@ -14,7 +14,8 @@ from PySide6.QtWidgets import (QApplication, QMainWindow, QVBoxLayout, QHBoxLayo
|
||||
QWidget, QPushButton, QTableWidget, QTableWidgetItem,
|
||||
QListWidget, QListWidgetItem, QSplitter, QFileDialog,
|
||||
QLabel, QStatusBar, QMessageBox, QHeaderView, QComboBox,
|
||||
QLineEdit, QGroupBox, QTextEdit, QStyledItemDelegate, QMenu)
|
||||
QLineEdit, QGroupBox, QTextEdit, QStyledItemDelegate, QMenu,
|
||||
QInputDialog)
|
||||
from PySide6.QtCore import Qt, QSize
|
||||
from PySide6.QtGui import QAction, QFontMetrics
|
||||
|
||||
@@ -244,6 +245,10 @@ class SQLiteViewer(QMainWindow):
|
||||
# 设置行头自动调整高度
|
||||
self.data_table.verticalHeader().setSectionResizeMode(QHeaderView.ResizeToContents)
|
||||
|
||||
# 启用行高调整功能 - 允许用户手动拖拽调整行高
|
||||
self.data_table.verticalHeader().setSectionsMovable(False) # 行不允许移动
|
||||
self.data_table.verticalHeader().setSectionResizeMode(QHeaderView.Interactive) # 允许手动调整行高
|
||||
|
||||
# 添加右键菜单支持
|
||||
self.data_table.setContextMenuPolicy(Qt.CustomContextMenu)
|
||||
self.data_table.customContextMenuRequested.connect(self.show_table_context_menu)
|
||||
@@ -572,6 +577,14 @@ class SQLiteViewer(QMainWindow):
|
||||
# 添加菜单项
|
||||
auto_resize_action = menu.addAction("自动调整列宽")
|
||||
auto_resize_rows_action = menu.addAction("自动调整行高")
|
||||
|
||||
# 添加行高调整子菜单
|
||||
row_height_menu = menu.addMenu("设置行高")
|
||||
increase_height_action = row_height_menu.addAction("增加行高 (+10px)")
|
||||
decrease_height_action = row_height_menu.addAction("减少行高 (-10px)")
|
||||
reset_height_action = row_height_menu.addAction("重置行高为默认值")
|
||||
custom_height_action = row_height_menu.addAction("自定义行高...")
|
||||
|
||||
copy_action = menu.addAction("复制选中内容")
|
||||
|
||||
# 显示菜单
|
||||
@@ -581,6 +594,14 @@ class SQLiteViewer(QMainWindow):
|
||||
self.auto_resize_columns()
|
||||
elif action == auto_resize_rows_action:
|
||||
self.auto_resize_rows()
|
||||
elif action == increase_height_action:
|
||||
self.adjust_row_height(10)
|
||||
elif action == decrease_height_action:
|
||||
self.adjust_row_height(-10)
|
||||
elif action == reset_height_action:
|
||||
self.reset_row_height()
|
||||
elif action == custom_height_action:
|
||||
self.set_custom_row_height()
|
||||
elif action == copy_action:
|
||||
self.copy_selected_content()
|
||||
|
||||
@@ -613,6 +634,68 @@ class SQLiteViewer(QMainWindow):
|
||||
|
||||
self.status_bar.showMessage("已自动调整行高")
|
||||
|
||||
def adjust_row_height(self, delta: int):
|
||||
"""调整选中行的行高"""
|
||||
selected_items = self.data_table.selectedItems()
|
||||
if not selected_items:
|
||||
# 如果没有选中行，调整所有行
|
||||
for row in range(self.data_table.rowCount()):
|
||||
current_height = self.data_table.rowHeight(row)
|
||||
new_height = max(current_height + delta, 20) # 最小行高20像素
|
||||
self.data_table.setRowHeight(row, new_height)
|
||||
self.status_bar.showMessage(f"所有行高已调整 {delta:+d} 像素")
|
||||
else:
|
||||
# 调整选中行
|
||||
selected_rows = set(item.row() for item in selected_items)
|
||||
for row in selected_rows:
|
||||
current_height = self.data_table.rowHeight(row)
|
||||
new_height = max(current_height + delta, 20) # 最小行高20像素
|
||||
self.data_table.setRowHeight(row, new_height)
|
||||
self.status_bar.showMessage(f"已调整 {len(selected_rows)} 行的行高 {delta:+d} 像素")
|
||||
|
||||
def reset_row_height(self):
|
||||
"""重置行高为默认值"""
|
||||
logger.info("重置行高为默认值")
|
||||
|
||||
# 重置为默认行高（30像素）
|
||||
default_height = 30
|
||||
for row in range(self.data_table.rowCount()):
|
||||
self.data_table.setRowHeight(row, default_height)
|
||||
|
||||
self.status_bar.showMessage("行高已重置为默认值")
|
||||
|
||||
def set_custom_row_height(self):
|
||||
"""设置自定义行高"""
|
||||
# 获取当前选中行的行高作为默认值
|
||||
selected_items = self.data_table.selectedItems()
|
||||
if selected_items:
|
||||
current_height = self.data_table.rowHeight(selected_items[0].row())
|
||||
else:
|
||||
current_height = 30
|
||||
|
||||
# 显示输入对话框
|
||||
height, ok = QInputDialog.getInt(
|
||||
self,
|
||||
"设置行高",
|
||||
"请输入行高（像素）:",
|
||||
current_height, # 默认值
|
||||
20, # 最小值
|
||||
500 # 最大值
|
||||
)
|
||||
|
||||
if ok:
|
||||
if selected_items:
|
||||
# 设置选中行
|
||||
selected_rows = set(item.row() for item in selected_items)
|
||||
for row in selected_rows:
|
||||
self.data_table.setRowHeight(row, height)
|
||||
self.status_bar.showMessage(f"已设置 {len(selected_rows)} 行的行高为 {height} 像素")
|
||||
else:
|
||||
# 设置所有行
|
||||
for row in range(self.data_table.rowCount()):
|
||||
self.data_table.setRowHeight(row, height)
|
||||
self.status_bar.showMessage(f"所有行高已设置为 {height} 像素")
|
||||
|
||||
def copy_selected_content(self):
|
||||
"""复制选中的内容"""
|
||||
selected_items = self.data_table.selectedItems()
|
||||
|
||||
166
product/web_sqlite_comparison.md
Normal file
166
product/web_sqlite_comparison.md
Normal file
@@ -0,0 +1,166 @@
|
||||
# Web SQLite查看器对比文档
|
||||
|
||||
## 概述
|
||||
|
||||
我为您创建了两个不同风格的Web SQLite数据库查看器，都支持现代化的界面、内容筛选和动态行高调整功能。
|
||||
|
||||
## 🚀 现代化版本 (modern_sqlite_viewer.py)
|
||||
|
||||
**访问地址**: http://localhost:5001
|
||||
|
||||
### 特点
|
||||
- **技术栈**: Flask + Bootstrap 5 + DataTables + jQuery
|
||||
- **界面风格**: 现代化渐变设计，卡片式布局
|
||||
- **功能特性**:
|
||||
- 服务器端分页处理，支持大数据集
|
||||
- 高级搜索和筛选功能
|
||||
- 响应式设计，完美适配移动端
|
||||
- 动态加载指示器
|
||||
- 专业的数据统计面板
|
||||
- 列排序和分页控制
|
||||
- 多行内容智能识别和美化显示
|
||||
|
||||
### 适用场景
|
||||
- 需要处理大量数据（数千条记录以上）
|
||||
- 需要专业级的数据分析和浏览功能
|
||||
- 需要移动端友好的界面
|
||||
- 需要高级的数据操作功能
|
||||
|
||||
## 🎯 轻量级版本 (simple_sqlite_viewer.py)
|
||||
|
||||
**访问地址**: http://localhost:5002
|
||||
|
||||
### 特点
|
||||
- **技术栈**: Flask + 纯原生HTML/CSS/JS（无外部依赖）
|
||||
- **界面风格**: 简洁优雅，内联样式
|
||||
- **功能特性**:
|
||||
- 客户端数据处理，快速响应
|
||||
- 轻量级搜索功能
|
||||
- 无外部依赖，加载速度快
|
||||
- 简洁的统计信息
|
||||
- 自适应行高显示
|
||||
- 移动端适配
|
||||
|
||||
### 适用场景
|
||||
- 数据量较小（几百条记录以内）
|
||||
- 需要快速部署和访问
|
||||
- 网络环境较差或需要离线使用
|
||||
- 偏好简洁无依赖的解决方案
|
||||
|
||||
## 🎨 共同特性
|
||||
|
||||
### 动态行高调整
|
||||
两个版本都实现了智能的行高调整：
|
||||
- **自动识别**: 自动检测多行文本内容
|
||||
- **美观显示**: 多行内容使用渐变背景和高亮边框
|
||||
- **自然变化**: 行高根据内容长度自然调整，无突兀感
|
||||
|
||||
### 内容筛选功能
|
||||
- **全局搜索**: 在所有列中搜索匹配内容
|
||||
- **实时筛选**: 输入时即时显示结果
|
||||
- **高亮显示**: 搜索结果清晰标识
|
||||
|
||||
### 数据库支持
|
||||
- **自动创建**: 如果product.db不存在，自动创建示例数据库
|
||||
- **表结构识别**: 自动识别所有表和列结构
|
||||
- **数据类型处理**: 智能处理各种数据类型（文本、数字、日期等）
|
||||
|
||||
## 🔧 技术实现亮点
|
||||
|
||||
### 多行内容处理
|
||||
```python
|
||||
# 智能检测和处理多行文本
|
||||
if isinstance(cell, str):
|
||||
if '\n' in cell:
|
||||
# 多行文本，用<br>替换换行符
|
||||
formatted_cell = cell.replace('\n', '<br>')
|
||||
row_data[col] = f'<div class="multiline-content">{formatted_cell}</div>'
|
||||
```
|
||||
|
||||
### 响应式设计
|
||||
- 使用CSS Grid和Flexbox布局
|
||||
- 媒体查询适配不同屏幕尺寸
|
||||
- 触摸友好的交互设计
|
||||
|
||||
### 性能优化
|
||||
- **防抖处理**: 搜索输入使用防抖技术
|
||||
- **异步加载**: 数据异步加载，界面不卡顿
|
||||
- **内存管理**: 合理的数据结构和内存使用
|
||||
|
||||
## 🚀 使用方法
|
||||
|
||||
### 启动应用
|
||||
```bash
|
||||
# 现代化版本
|
||||
python modern_sqlite_viewer.py
|
||||
|
||||
# 轻量级版本
|
||||
python simple_sqlite_viewer.py
|
||||
```
|
||||
|
||||
### 访问界面
|
||||
- 打开浏览器访问对应地址
|
||||
- 选择要查看的数据表
|
||||
- 使用搜索框筛选内容
|
||||
- 观察行高随内容自然变化
|
||||
|
||||
### 数据文件
|
||||
- 默认读取 `product.db` 文件
|
||||
- 如果文件不存在，自动创建示例数据
|
||||
- 支持任何SQLite数据库文件
|
||||
|
||||
## 🎨 界面预览
|
||||
|
||||
### 现代化版本界面特色
|
||||
- 渐变背景和毛玻璃效果
|
||||
- 卡片式布局设计
|
||||
- 专业的数据统计面板
|
||||
- 高级DataTables功能
|
||||
|
||||
### 轻量级版本界面特色
|
||||
- 简洁的渐变设计
|
||||
- 内联CSS，无外部依赖
|
||||
- 快速响应的用户体验
|
||||
- 清晰的统计信息
|
||||
|
||||
## 📊 性能对比
|
||||
|
||||
| 特性 | 现代化版本 | 轻量级版本 |
|
||||
|------|------------|------------|
|
||||
| 加载速度 | 中等（依赖CDN） | 快速（内联资源） |
|
||||
| 数据处理能力 | 强（服务器端分页） | 中（客户端处理） |
|
||||
| 界面美观度 | 高（Bootstrap 5） | 中（自定义样式） |
|
||||
| 功能丰富度 | 高（DataTables） | 中（基础功能） |
|
||||
| 外部依赖 | 有（CDN资源） | 无（纯内联） |
|
||||
| 移动端适配 | 优秀 | 良好 |
|
||||
|
||||
## 🔧 自定义和扩展
|
||||
|
||||
### 修改数据库路径
|
||||
在代码中修改 `DB_PATH` 变量即可：
|
||||
```python
|
||||
DB_PATH = "path/to/your/database.db"
|
||||
```
|
||||
|
||||
### 添加新功能
|
||||
两个版本都基于模块化设计，易于扩展：
|
||||
- 添加新的API端点
|
||||
- 自定义界面样式
|
||||
- 增加数据处理逻辑
|
||||
|
||||
### 部署到生产环境
|
||||
建议使用生产级WSGI服务器：
|
||||
```bash
|
||||
pip install gunicorn
|
||||
gunicorn -w 4 -b 0.0.0.0:5000 your_app:app
|
||||
```
|
||||
|
||||
## 🎯 选择建议
|
||||
|
||||
- **需要专业功能** → 选择现代化版本
|
||||
- **需要快速部署** → 选择轻量级版本
|
||||
- **数据量较大** → 选择现代化版本
|
||||
- **网络环境差** → 选择轻量级版本
|
||||
- **需要离线使用** → 选择轻量级版本
|
||||
|
||||
两个版本都提供了优秀的用户体验和现代化的界面设计，您可以根据具体需求选择合适的版本。
|
||||
741
product/web_sqlite_viewer.py
Normal file
741
product/web_sqlite_viewer.py
Normal file
@@ -0,0 +1,741 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
现代化Web SQLite数据库查看器
|
||||
基于Flask框架，提供自然的内容显示和筛选功能
|
||||
"""
|
||||
|
||||
from flask import Flask, render_template, jsonify, request, send_from_directory
|
||||
import sqlite3
|
||||
import os
|
||||
import json
|
||||
from datetime import datetime
|
||||
from loguru import logger
|
||||
|
||||
app = Flask(__name__)
|
||||
|
||||
# 数据库路径
|
||||
DB_PATH = os.path.join(os.path.dirname(__file__), 'product.db')
|
||||
|
||||
class SQLiteWebViewer:
|
||||
def __init__(self, db_path):
|
||||
self.db_path = db_path
|
||||
logger.info(f"初始化Web SQLite查看器，数据库路径: {db_path}")
|
||||
|
||||
def get_tables(self):
|
||||
"""获取所有表名"""
|
||||
try:
|
||||
conn = sqlite3.connect(self.db_path)
|
||||
cursor = conn.cursor()
|
||||
cursor.execute("SELECT name FROM sqlite_master WHERE type='table' AND name NOT LIKE 'sqlite_%'")
|
||||
tables = [row[0] for row in cursor.fetchall()]
|
||||
conn.close()
|
||||
logger.info(f"获取到 {len(tables)} 个表")
|
||||
return tables
|
||||
except Exception as e:
|
||||
logger.error(f"获取表列表失败: {e}")
|
||||
return []
|
||||
|
||||
def get_table_structure(self, table_name):
|
||||
"""获取表结构"""
|
||||
try:
|
||||
conn = sqlite3.connect(self.db_path)
|
||||
cursor = conn.cursor()
|
||||
cursor.execute(f"PRAGMA table_info({table_name})")
|
||||
columns = cursor.fetchall()
|
||||
conn.close()
|
||||
|
||||
structure = []
|
||||
for col in columns:
|
||||
structure.append({
|
||||
'cid': col[0],
|
||||
'name': col[1],
|
||||
'type': col[2],
|
||||
'notnull': col[3],
|
||||
'default': col[4],
|
||||
'pk': col[5]
|
||||
})
|
||||
|
||||
logger.info(f"获取表 {table_name} 结构，共 {len(structure)} 个字段")
|
||||
return structure
|
||||
except Exception as e:
|
||||
logger.error(f"获取表结构失败: {e}")
|
||||
return []
|
||||
|
||||
def get_table_data(self, table_name, page=1, per_page=50, search_field=None, search_value=None):
|
||||
"""获取表数据，支持分页和搜索"""
|
||||
try:
|
||||
conn = sqlite3.connect(self.db_path)
|
||||
cursor = conn.cursor()
|
||||
|
||||
# 获取总记录数
|
||||
if search_field and search_value:
|
||||
count_query = f"SELECT COUNT(*) FROM {table_name} WHERE {search_field} LIKE ?"
|
||||
cursor.execute(count_query, (f'%{search_value}%',))
|
||||
else:
|
||||
count_query = f"SELECT COUNT(*) FROM {table_name}"
|
||||
cursor.execute(count_query)
|
||||
|
||||
total_count = cursor.fetchone()[0]
|
||||
|
||||
# 获取分页数据
|
||||
offset = (page - 1) * per_page
|
||||
|
||||
if search_field and search_value:
|
||||
data_query = f"SELECT * FROM {table_name} WHERE {search_field} LIKE ? LIMIT ? OFFSET ?"
|
||||
cursor.execute(data_query, (f'%{search_value}%', per_page, offset))
|
||||
else:
|
||||
data_query = f"SELECT * FROM {table_name} LIMIT ? OFFSET ?"
|
||||
cursor.execute(data_query, (per_page, offset))
|
||||
|
||||
rows = cursor.fetchall()
|
||||
|
||||
# 获取列名
|
||||
cursor.execute(f"PRAGMA table_info({table_name})")
|
||||
columns = [col[1] for col in cursor.fetchall()]
|
||||
|
||||
conn.close()
|
||||
|
||||
# 处理数据，检测多行文本
|
||||
processed_rows = []
|
||||
for row in rows:
|
||||
processed_row = []
|
||||
for cell in row:
|
||||
if cell is None:
|
||||
processed_row.append({'value': '', 'type': 'empty'})
|
||||
elif isinstance(cell, str) and ('\n' in cell or len(cell) > 100):
|
||||
# 多行文本或长文本
|
||||
lines = cell.count('\n') + 1
|
||||
processed_row.append({
|
||||
'value': cell,
|
||||
'type': 'multiline',
|
||||
'lines': lines,
|
||||
'length': len(cell)
|
||||
})
|
||||
else:
|
||||
processed_row.append({'value': str(cell), 'type': 'normal'})
|
||||
processed_rows.append(processed_row)
|
||||
|
||||
logger.info(f"获取表 {table_name} 数据，第 {page} 页，共 {len(processed_rows)} 条记录")
|
||||
return {
|
||||
'columns': columns,
|
||||
'rows': processed_rows,
|
||||
'total_count': total_count,
|
||||
'page': page,
|
||||
'per_page': per_page,
|
||||
'total_pages': (total_count + per_page - 1) // per_page
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"获取表数据失败: {e}")
|
||||
return {'columns': [], 'rows': [], 'total_count': 0, 'page': 1, 'per_page': 50, 'total_pages': 0}
|
||||
|
||||
# 初始化查看器
|
||||
viewer = SQLiteWebViewer(DB_PATH)
|
||||
|
||||
@app.route('/')
|
||||
def index():
|
||||
"""主页"""
|
||||
logger.info("访问主页")
|
||||
return render_template('index.html')
|
||||
|
||||
@app.route('/api/tables')
|
||||
def get_tables():
|
||||
"""获取所有表"""
|
||||
tables = viewer.get_tables()
|
||||
return jsonify({'tables': tables})
|
||||
|
||||
@app.route('/api/table/<table_name>/structure')
|
||||
def get_table_structure(table_name):
|
||||
"""获取表结构"""
|
||||
structure = viewer.get_table_structure(table_name)
|
||||
return jsonify({'structure': structure})
|
||||
|
||||
@app.route('/api/table/<table_name>/data')
|
||||
def get_table_data(table_name):
|
||||
"""获取表数据"""
|
||||
page = int(request.args.get('page', 1))
|
||||
per_page = int(request.args.get('per_page', 50))
|
||||
search_field = request.args.get('search_field')
|
||||
search_value = request.args.get('search_value')
|
||||
|
||||
data = viewer.get_table_data(table_name, page, per_page, search_field, search_value)
|
||||
return jsonify(data)
|
||||
|
||||
@app.route('/static/<path:filename>')
|
||||
def static_files(filename):
|
||||
"""静态文件"""
|
||||
return send_from_directory('static', filename)
|
||||
|
||||
def create_html_template():
|
||||
"""创建HTML模板"""
|
||||
template_dir = 'templates'
|
||||
if not os.path.exists(template_dir):
|
||||
os.makedirs(template_dir)
|
||||
|
||||
html_content = '''<!DOCTYPE html>
|
||||
<html lang="zh-CN">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>SQLite数据库查看器</title>
|
||||
<style>
|
||||
* {
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
body {
|
||||
font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
|
||||
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
|
||||
min-height: 100vh;
|
||||
color: #333;
|
||||
}
|
||||
|
||||
.container {
|
||||
max-width: 1400px;
|
||||
margin: 0 auto;
|
||||
padding: 20px;
|
||||
}
|
||||
|
||||
.header {
|
||||
background: rgba(255, 255, 255, 0.95);
|
||||
backdrop-filter: blur(10px);
|
||||
border-radius: 15px;
|
||||
padding: 25px;
|
||||
margin-bottom: 25px;
|
||||
box-shadow: 0 8px 32px rgba(0, 0, 0, 0.1);
|
||||
}
|
||||
|
||||
.header h1 {
|
||||
color: #2c3e50;
|
||||
font-size: 2.5em;
|
||||
margin-bottom: 10px;
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
.controls {
|
||||
display: flex;
|
||||
gap: 20px;
|
||||
align-items: center;
|
||||
flex-wrap: wrap;
|
||||
margin-top: 20px;
|
||||
}
|
||||
|
||||
.control-group {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 8px;
|
||||
}
|
||||
|
||||
.control-group label {
|
||||
font-weight: 600;
|
||||
color: #34495e;
|
||||
font-size: 0.9em;
|
||||
}
|
||||
|
||||
select, input {
|
||||
padding: 12px 15px;
|
||||
border: 2px solid #e0e6ed;
|
||||
border-radius: 8px;
|
||||
font-size: 14px;
|
||||
transition: all 0.3s ease;
|
||||
background: white;
|
||||
}
|
||||
|
||||
select:focus, input:focus {
|
||||
outline: none;
|
||||
border-color: #667eea;
|
||||
box-shadow: 0 0 0 3px rgba(102, 126, 234, 0.1);
|
||||
}
|
||||
|
||||
.btn {
|
||||
padding: 12px 24px;
|
||||
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
|
||||
color: white;
|
||||
border: none;
|
||||
border-radius: 8px;
|
||||
cursor: pointer;
|
||||
font-weight: 600;
|
||||
transition: all 0.3s ease;
|
||||
text-decoration: none;
|
||||
display: inline-block;
|
||||
}
|
||||
|
||||
.btn:hover {
|
||||
transform: translateY(-2px);
|
||||
box-shadow: 0 5px 15px rgba(0, 0, 0, 0.2);
|
||||
}
|
||||
|
||||
.data-container {
|
||||
background: rgba(255, 255, 255, 0.95);
|
||||
backdrop-filter: blur(10px);
|
||||
border-radius: 15px;
|
||||
padding: 25px;
|
||||
box-shadow: 0 8px 32px rgba(0, 0, 0, 0.1);
|
||||
}
|
||||
|
||||
.table-info {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
margin-bottom: 20px;
|
||||
padding: 15px;
|
||||
background: #f8f9fa;
|
||||
border-radius: 10px;
|
||||
}
|
||||
|
||||
.table-info h2 {
|
||||
color: #2c3e50;
|
||||
font-size: 1.5em;
|
||||
}
|
||||
|
||||
.stats {
|
||||
display: flex;
|
||||
gap: 20px;
|
||||
font-size: 0.9em;
|
||||
color: #7f8c8d;
|
||||
}
|
||||
|
||||
.table-wrapper {
|
||||
overflow-x: auto;
|
||||
border-radius: 10px;
|
||||
box-shadow: 0 4px 16px rgba(0, 0, 0, 0.1);
|
||||
}
|
||||
|
||||
table {
|
||||
width: 100%;
|
||||
border-collapse: collapse;
|
||||
background: white;
|
||||
font-size: 14px;
|
||||
}
|
||||
|
||||
th {
|
||||
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
|
||||
color: white;
|
||||
padding: 15px 12px;
|
||||
text-align: left;
|
||||
font-weight: 600;
|
||||
position: sticky;
|
||||
top: 0;
|
||||
z-index: 10;
|
||||
}
|
||||
|
||||
td {
|
||||
padding: 12px;
|
||||
border-bottom: 1px solid #ecf0f1;
|
||||
vertical-align: top;
|
||||
}
|
||||
|
||||
tr:nth-child(even) {
|
||||
background-color: #f8f9fa;
|
||||
}
|
||||
|
||||
tr:hover {
|
||||
background-color: #e3f2fd;
|
||||
transition: background-color 0.3s ease;
|
||||
}
|
||||
|
||||
.multiline-cell {
|
||||
white-space: pre-wrap;
|
||||
line-height: 1.6;
|
||||
max-height: 200px;
|
||||
overflow-y: auto;
|
||||
padding: 8px;
|
||||
background: #fff3cd;
|
||||
border-radius: 6px;
|
||||
border-left: 4px solid #ffc107;
|
||||
}
|
||||
|
||||
.normal-cell {
|
||||
white-space: nowrap;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
max-width: 300px;
|
||||
}
|
||||
|
||||
.empty-cell {
|
||||
color: #95a5a6;
|
||||
font-style: italic;
|
||||
}
|
||||
|
||||
.pagination {
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
gap: 10px;
|
||||
margin-top: 25px;
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
|
||||
.page-info {
|
||||
color: #7f8c8d;
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.page-btn {
|
||||
padding: 8px 12px;
|
||||
border: 2px solid #e0e6ed;
|
||||
background: white;
|
||||
border-radius: 6px;
|
||||
cursor: pointer;
|
||||
transition: all 0.3s ease;
|
||||
min-width: 40px;
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
.page-btn:hover {
|
||||
border-color: #667eea;
|
||||
background: #667eea;
|
||||
color: white;
|
||||
}
|
||||
|
||||
.page-btn.active {
|
||||
background: #667eea;
|
||||
color: white;
|
||||
border-color: #667eea;
|
||||
}
|
||||
|
||||
.page-btn:disabled {
|
||||
opacity: 0.5;
|
||||
cursor: not-allowed;
|
||||
}
|
||||
|
||||
.loading {
|
||||
text-align: center;
|
||||
padding: 40px;
|
||||
color: #7f8c8d;
|
||||
font-size: 1.1em;
|
||||
}
|
||||
|
||||
.error {
|
||||
background: #f8d7da;
|
||||
color: #721c24;
|
||||
padding: 15px;
|
||||
border-radius: 8px;
|
||||
border: 1px solid #f5c6cb;
|
||||
margin: 20px 0;
|
||||
}
|
||||
|
||||
.no-data {
|
||||
text-align: center;
|
||||
padding: 40px;
|
||||
color: #7f8c8d;
|
||||
font-size: 1.1em;
|
||||
}
|
||||
|
||||
@media (max-width: 768px) {
|
||||
.controls {
|
||||
flex-direction: column;
|
||||
align-items: stretch;
|
||||
}
|
||||
|
||||
.table-info {
|
||||
flex-direction: column;
|
||||
gap: 15px;
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
.stats {
|
||||
justify-content: center;
|
||||
}
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="container">
|
||||
<div class="header">
|
||||
<h1>🗄️ SQLite数据库查看器</h1>
|
||||
<div class="controls">
|
||||
<div class="control-group">
|
||||
<label for="tableSelect">选择数据表:</label>
|
||||
<select id="tableSelect">
|
||||
<option value="">加载中...</option>
|
||||
</select>
|
||||
</div>
|
||||
<div class="control-group">
|
||||
<label for="searchField">筛选字段:</label>
|
||||
<select id="searchField" disabled>
|
||||
<option value="">选择字段...</option>
|
||||
</select>
|
||||
</div>
|
||||
<div class="control-group">
|
||||
<label for="searchValue">筛选内容:</label>
|
||||
<input type="text" id="searchValue" placeholder="输入筛选内容..." disabled>
|
||||
</div>
|
||||
<button class="btn" onclick="loadData()">刷新数据</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="data-container">
|
||||
<div class="table-info">
|
||||
<h2 id="tableName">请选择数据表</h2>
|
||||
<div class="stats">
|
||||
<span id="recordCount">记录数: 0</span>
|
||||
<span id="pageInfo">第 0 页，共 0 页</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="dataContainer">
|
||||
<div class="no-data">请选择数据表以查看内容</div>
|
||||
</div>
|
||||
|
||||
<div id="pagination" class="pagination" style="display: none;">
|
||||
<button class="page-btn" onclick="changePage('prev')" id="prevBtn">上一页</button>
|
||||
<span class="page-info" id="pageInfoDetail"></span>
|
||||
<button class="page-btn" onclick="changePage('next')" id="nextBtn">下一页</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
let currentTable = '';
|
||||
let currentPage = 1;
|
||||
let perPage = 50;
|
||||
let totalPages = 1;
|
||||
let currentData = null;
|
||||
|
||||
// 初始化
|
||||
document.addEventListener('DOMContentLoaded', function() {
|
||||
loadTables();
|
||||
|
||||
// 绑定事件
|
||||
document.getElementById('tableSelect').addEventListener('change', function() {
|
||||
currentTable = this.value;
|
||||
currentPage = 1;
|
||||
if (currentTable) {
|
||||
loadTableStructure();
|
||||
loadData();
|
||||
}
|
||||
});
|
||||
|
||||
document.getElementById('searchField').addEventListener('change', loadData);
|
||||
document.getElementById('searchValue').addEventListener('input', debounce(loadData, 500));
|
||||
});
|
||||
|
||||
// 防抖函数
|
||||
function debounce(func, wait) {
|
||||
let timeout;
|
||||
return function executedFunction(...args) {
|
||||
const later = () => {
|
||||
clearTimeout(timeout);
|
||||
func(...args);
|
||||
};
|
||||
clearTimeout(timeout);
|
||||
timeout = setTimeout(later, wait);
|
||||
};
|
||||
}
|
||||
|
||||
// 加载表列表
|
||||
async function loadTables() {
|
||||
try {
|
||||
const response = await fetch('/api/tables');
|
||||
const data = await response.json();
|
||||
const select = document.getElementById('tableSelect');
|
||||
select.innerHTML = '<option value="">选择数据表...</option>';
|
||||
|
||||
data.tables.forEach(table => {
|
||||
const option = document.createElement('option');
|
||||
option.value = table;
|
||||
option.textContent = table;
|
||||
select.appendChild(option);
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('加载表列表失败:', error);
|
||||
showError('加载表列表失败: ' + error.message);
|
||||
}
|
||||
}
|
||||
|
||||
// 加载表结构
|
||||
async function loadTableStructure() {
|
||||
if (!currentTable) return;
|
||||
|
||||
try {
|
||||
const response = await fetch(`/api/table/${currentTable}/structure`);
|
||||
const data = await response.json();
|
||||
const searchField = document.getElementById('searchField');
|
||||
|
||||
searchField.innerHTML = '<option value="">所有字段</option>';
|
||||
data.structure.forEach(field => {
|
||||
const option = document.createElement('option');
|
||||
option.value = field.name;
|
||||
option.textContent = field.name;
|
||||
searchField.appendChild(option);
|
||||
});
|
||||
|
||||
searchField.disabled = false;
|
||||
document.getElementById('searchValue').disabled = false;
|
||||
} catch (error) {
|
||||
console.error('加载表结构失败:', error);
|
||||
}
|
||||
}
|
||||
|
||||
// 加载数据
|
||||
async function loadData() {
|
||||
if (!currentTable) return;
|
||||
|
||||
const container = document.getElementById('dataContainer');
|
||||
container.innerHTML = '<div class="loading">📊 数据加载中...</div>';
|
||||
|
||||
const searchField = document.getElementById('searchField').value;
|
||||
const searchValue = document.getElementById('searchValue').value;
|
||||
|
||||
try {
|
||||
let url = `/api/table/${currentTable}/data?page=${currentPage}&per_page=${perPage}`;
|
||||
if (searchField && searchValue) {
|
||||
url += `&search_field=${searchField}&search_value=${encodeURIComponent(searchValue)}`;
|
||||
}
|
||||
|
||||
const response = await fetch(url);
|
||||
currentData = await response.json();
|
||||
|
||||
displayData(currentData);
|
||||
updatePagination();
|
||||
|
||||
} catch (error) {
|
||||
console.error('加载数据失败:', error);
|
||||
showError('加载数据失败: ' + error.message);
|
||||
}
|
||||
}
|
||||
|
||||
// 显示数据
|
||||
function displayData(data) {
|
||||
const container = document.getElementById('dataContainer');
|
||||
|
||||
if (!data.rows || data.rows.length === 0) {
|
||||
container.innerHTML = '<div class="no-data">📭 没有找到数据</div>';
|
||||
return;
|
||||
}
|
||||
|
||||
let html = '<div class="table-wrapper"><table><thead><tr>';
|
||||
|
||||
// 表头
|
||||
data.columns.forEach(col => {
|
||||
html += `<th>${col}</th>`;
|
||||
});
|
||||
html += '</tr></thead><tbody>';
|
||||
|
||||
// 数据行
|
||||
data.rows.forEach(row => {
|
||||
html += '<tr>';
|
||||
row.forEach(cell => {
|
||||
if (cell.type === 'multiline') {
|
||||
html += `<td><div class="multiline-cell">${escapeHtml(cell.value)}</div></td>`;
|
||||
} else if (cell.type === 'empty') {
|
||||
html += '<td><div class="empty-cell">空</div></td>';
|
||||
} else {
|
||||
html += `<td><div class="normal-cell">${escapeHtml(cell.value)}</div></td>`;
|
||||
}
|
||||
});
|
||||
html += '</tr>';
|
||||
});
|
||||
|
||||
html += '</tbody></table></div>';
|
||||
container.innerHTML = html;
|
||||
|
||||
// 更新统计信息
|
||||
document.getElementById('tableName').textContent = `📋 ${currentTable}`;
|
||||
document.getElementById('recordCount').textContent = `记录数: ${data.total_count}`;
|
||||
document.getElementById('pageInfo').textContent = `第 ${currentPage} 页，共 ${data.total_pages} 页`;
|
||||
}
|
||||
|
||||
// 更新分页
|
||||
function updatePagination() {
|
||||
if (!currentData) return;
|
||||
|
||||
totalPages = currentData.total_pages;
|
||||
const pagination = document.getElementById('pagination');
|
||||
const prevBtn = document.getElementById('prevBtn');
|
||||
const nextBtn = document.getElementById('nextBtn');
|
||||
const pageInfo = document.getElementById('pageInfoDetail');
|
||||
|
||||
if (totalPages <= 1) {
|
||||
pagination.style.display = 'none';
|
||||
return;
|
||||
}
|
||||
|
||||
pagination.style.display = 'flex';
|
||||
|
||||
prevBtn.disabled = currentPage <= 1;
|
||||
nextBtn.disabled = currentPage >= totalPages;
|
||||
|
||||
pageInfo.textContent = `${currentPage} / ${totalPages}`;
|
||||
}
|
||||
|
||||
// 翻页
|
||||
function changePage(direction) {
|
||||
if (direction === 'prev' && currentPage > 1) {
|
||||
currentPage--;
|
||||
loadData();
|
||||
} else if (direction === 'next' && currentPage < totalPages) {
|
||||
currentPage++;
|
||||
loadData();
|
||||
}
|
||||
}
|
||||
|
||||
// HTML转义
|
||||
function escapeHtml(text) {
|
||||
const div = document.createElement('div');
|
||||
div.textContent = text;
|
||||
return div.innerHTML;
|
||||
}
|
||||
|
||||
// 显示错误
|
||||
function showError(message) {
|
||||
const container = document.getElementById('dataContainer');
|
||||
container.innerHTML = `<div class="error">❌ ${message}</div>`;
|
||||
}
|
||||
</script>
|
||||
</body>
|
||||
</html>'''
|
||||
|
||||
with open(os.path.join(template_dir, 'index.html'), 'w', encoding='utf-8') as f:
|
||||
f.write(html_content)
|
||||
|
||||
logger.info("HTML模板创建完成")
|
||||
|
||||
if __name__ == '__main__':
|
||||
logger.info("启动Web SQLite查看器")
|
||||
|
||||
# 创建HTML模板
|
||||
create_html_template()
|
||||
|
||||
# 检查数据库文件
|
||||
if not os.path.exists(DB_PATH):
|
||||
logger.warning(f"数据库文件不存在: {DB_PATH}")
|
||||
logger.info("将创建一个空的数据库用于演示")
|
||||
|
||||
# 创建示例数据库
|
||||
conn = sqlite3.connect(DB_PATH)
|
||||
cursor = conn.cursor()
|
||||
|
||||
# 创建示例表
|
||||
cursor.execute('''
|
||||
CREATE TABLE IF NOT EXISTS products (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
name TEXT NOT NULL,
|
||||
description TEXT,
|
||||
features TEXT,
|
||||
price REAL,
|
||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
||||
)
|
||||
''')
|
||||
|
||||
# 插入示例数据
|
||||
sample_data = [
|
||||
('产品A', '这是一个非常优秀的产品\n具有多种实用功能\n用户反馈很好', '高性能\n易用性\n稳定性', 99.99),
|
||||
('产品B', '创新设计\n简洁界面\n强大功能', '创新\n美观\n实用', 149.99),
|
||||
('产品C', '专业级解决方案\n适用于企业环境\n支持大规模部署', '企业级\n可扩展\n安全', 299.99)
|
||||
]
|
||||
|
||||
cursor.executemany('INSERT INTO products (name, description, features, price) VALUES (?, ?, ?, ?)', sample_data)
|
||||
conn.commit()
|
||||
conn.close()
|
||||
|
||||
logger.info("示例数据库创建完成")
|
||||
|
||||
logger.info(f"Web服务器启动，访问地址: http://localhost:5000")
|
||||
logger.info("按 Ctrl+C 停止服务器")
|
||||
|
||||
app.run(debug=True, host='0.0.0.0', port=5000)
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 477 KiB |
@@ -1,42 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
playwright_stealth 使用示例
|
||||
演示如何使用 stealth 模式运行 ProductHunt 爬虫
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
from loguru import logger
|
||||
from product.new_data_stealth import ProductHuntScraper
|
||||
|
||||
async def run_stealth_scraper():
|
||||
"""运行 stealth 版本的爬虫"""
|
||||
logger.info("开始运行 stealth 版本的 ProductHunt 爬虫")
|
||||
|
||||
# 创建爬虫实例
|
||||
scraper = ProductHuntScraper()
|
||||
|
||||
# 执行爬取
|
||||
success = await scraper.scrape()
|
||||
|
||||
if success:
|
||||
logger.success("Stealth 爬虫执行成功！")
|
||||
logger.info("生成的文件：")
|
||||
logger.info("- product_info_stealth.json: 产品信息数据")
|
||||
logger.info("- product_page_stealth.html: 页面HTML内容")
|
||||
logger.info("- product_screenshot_stealth.png: 页面截图")
|
||||
else:
|
||||
logger.error("Stealth 爬虫执行失败")
|
||||
|
||||
return success
|
||||
|
||||
def main():
|
||||
"""主函数"""
|
||||
logger.info("=== playwright_stealth 使用示例 ===")
|
||||
logger.info("此示例演示如何使用 playwright_stealth 模块增强浏览器反检测能力")
|
||||
|
||||
# 运行异步任务
|
||||
asyncio.run(run_stealth_scraper())
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,91 +0,0 @@
|
||||
2025-11-26 23:10:59.326 | INFO | __main__:__init__:27 - 初始化SQLite数据库查看器
|
||||
2025-11-26 23:10:59.327 | INFO | __main__:init_ui:34 - 设置主窗口界面
|
||||
2025-11-26 23:10:59.327 | INFO | __main__:create_top_buttons:64 - 创建顶部按钮
|
||||
2025-11-26 23:10:59.327 | INFO | __main__:create_filter_section:87 - 创建筛选控件区域
|
||||
2025-11-26 23:10:59.334 | INFO | __main__:create_splitter:132 - 创建分割器界面
|
||||
2025-11-26 23:10:59.335 | INFO | __main__:create_status_bar:161 - 创建状态栏
|
||||
2025-11-26 23:10:59.335 | INFO | __main__:create_menubar:168 - 创建菜单栏
|
||||
2025-11-26 23:10:59.344 | INFO | __main__:init_ui:60 - 界面初始化完成
|
||||
2025-11-26 23:10:59.494 | INFO | __main__:main:426 - 应用程序启动完成
|
||||
2025-11-26 23:11:01.008 | INFO | __main__:open_database:184 - 打开数据库文件对话框
|
||||
2025-11-26 23:11:03.377 | INFO | __main__:open_database:198 - 打开数据库文件: C:/Users/xiaji/Documents/个人文件夹/夏骥/hothub的抓取/product/products.db
|
||||
2025-11-26 23:11:03.378 | INFO | __main__:connect_to_database:208 - 数据库连接成功
|
||||
2025-11-26 23:11:03.379 | INFO | __main__:load_table_list:236 - 加载了 3 个表
|
||||
2025-11-26 23:11:06.700 | INFO | __main__:on_table_selected:246 - 选中表: product_analysis
|
||||
2025-11-26 23:11:06.701 | INFO | __main__:load_table_data:282 - 加载表 product_analysis 数据完成，共 5 行
|
||||
2025-11-26 23:11:06.701 | INFO | __main__:update_field_combo:312 - 更新字段下拉框: product_analysis, 共 8 个字段
|
||||
2025-11-26 23:11:49.497 | INFO | __main__:closeEvent:404 - 关闭应用程序
|
||||
2025-11-27 21:17:24.999 | INFO | __main__:__init__:27 - 初始化SQLite数据库查看器
|
||||
2025-11-27 21:17:25.000 | INFO | __main__:init_ui:34 - 设置主窗口界面
|
||||
2025-11-27 21:17:25.002 | INFO | __main__:create_top_buttons:64 - 创建顶部按钮
|
||||
2025-11-27 21:17:25.007 | INFO | __main__:create_filter_section:87 - 创建筛选控件区域
|
||||
2025-11-27 21:17:25.022 | INFO | __main__:create_splitter:132 - 创建分割器界面
|
||||
2025-11-27 21:17:25.036 | INFO | __main__:create_status_bar:161 - 创建状态栏
|
||||
2025-11-27 21:17:25.038 | INFO | __main__:create_menubar:168 - 创建菜单栏
|
||||
2025-11-27 21:17:25.061 | INFO | __main__:init_ui:60 - 界面初始化完成
|
||||
2025-11-27 21:17:25.250 | INFO | __main__:main:426 - 应用程序启动完成
|
||||
2025-11-27 21:17:28.396 | INFO | __main__:open_database:184 - 打开数据库文件对话框
|
||||
2025-11-27 21:17:29.780 | INFO | __main__:open_database:198 - 打开数据库文件: C:/Users/xiaji/Documents/个人文件夹/夏骥/hothub的抓取/tophub_data.db
|
||||
2025-11-27 21:17:29.786 | INFO | __main__:connect_to_database:208 - 数据库连接成功
|
||||
2025-11-27 21:17:29.792 | INFO | __main__:load_table_list:236 - 加载了 2 个表
|
||||
2025-11-27 21:17:33.220 | INFO | __main__:on_table_selected:246 - 选中表: articles
|
||||
2025-11-27 21:17:33.539 | INFO | __main__:load_table_data:282 - 加载表 articles 数据完成，共 16942 行
|
||||
2025-11-27 21:17:33.542 | INFO | __main__:update_field_combo:312 - 更新字段下拉框: articles, 共 8 个字段
|
||||
2025-11-27 21:17:39.131 | INFO | __main__:closeEvent:404 - 关闭应用程序
|
||||
2025-11-27 22:10:53.621 | INFO | __main__:__init__:27 - 初始化SQLite数据库查看器
|
||||
2025-11-27 22:10:53.622 | INFO | __main__:init_ui:34 - 设置主窗口界面
|
||||
2025-11-27 22:10:53.624 | INFO | __main__:create_top_buttons:64 - 创建顶部按钮
|
||||
2025-11-27 22:10:53.629 | INFO | __main__:create_filter_section:87 - 创建筛选控件区域
|
||||
2025-11-27 22:10:53.643 | INFO | __main__:create_splitter:132 - 创建分割器界面
|
||||
2025-11-27 22:10:53.656 | INFO | __main__:create_status_bar:161 - 创建状态栏
|
||||
2025-11-27 22:10:53.658 | INFO | __main__:create_menubar:168 - 创建菜单栏
|
||||
2025-11-27 22:10:53.678 | INFO | __main__:init_ui:60 - 界面初始化完成
|
||||
2025-11-27 22:10:53.867 | INFO | __main__:main:426 - 应用程序启动完成
|
||||
2025-11-27 22:10:59.059 | INFO | __main__:open_database:184 - 打开数据库文件对话框
|
||||
2025-11-27 22:11:00.561 | INFO | __main__:open_database:198 - 打开数据库文件: C:/Users/xiaji/Documents/个人文件夹/夏骥/hothub的抓取/tophub_data.db
|
||||
2025-11-27 22:11:00.562 | INFO | __main__:connect_to_database:208 - 数据库连接成功
|
||||
2025-11-27 22:11:00.563 | INFO | __main__:load_table_list:236 - 加载了 2 个表
|
||||
2025-11-27 22:11:02.513 | INFO | __main__:on_table_selected:246 - 选中表: articles
|
||||
2025-11-27 22:11:02.853 | INFO | __main__:load_table_data:282 - 加载表 articles 数据完成，共 16942 行
|
||||
2025-11-27 22:11:02.857 | INFO | __main__:update_field_combo:312 - 更新字段下拉框: articles, 共 8 个字段
|
||||
2025-11-27 22:11:19.598 | INFO | __main__:open_database:184 - 打开数据库文件对话框
|
||||
2025-11-27 22:11:22.448 | INFO | __main__:open_database:198 - 打开数据库文件: C:/Users/xiaji/Documents/个人文件夹/夏骥/hothub的抓取/product/products.db
|
||||
2025-11-27 22:11:22.449 | INFO | __main__:connect_to_database:208 - 数据库连接成功
|
||||
2025-11-27 22:11:22.452 | INFO | __main__:load_table_list:236 - 加载了 3 个表
|
||||
2025-11-27 22:11:24.895 | INFO | __main__:on_table_selected:246 - 选中表: product_analysis
|
||||
2025-11-27 22:11:25.053 | INFO | __main__:load_table_data:282 - 加载表 product_analysis 数据完成，共 251 行
|
||||
2025-11-27 22:11:25.054 | INFO | __main__:update_field_combo:312 - 更新字段下拉框: product_analysis, 共 8 个字段
|
||||
2025-11-27 22:14:44.131 | INFO | __main__:closeEvent:404 - 关闭应用程序
|
||||
2025-11-27 22:48:07.339 | INFO | __main__:__init__:27 - 初始化SQLite数据库查看器
|
||||
2025-11-27 22:48:07.340 | INFO | __main__:init_ui:34 - 设置主窗口界面
|
||||
2025-11-27 22:48:07.342 | INFO | __main__:create_top_buttons:64 - 创建顶部按钮
|
||||
2025-11-27 22:48:07.347 | INFO | __main__:create_filter_section:87 - 创建筛选控件区域
|
||||
2025-11-27 22:48:07.363 | INFO | __main__:create_splitter:132 - 创建分割器界面
|
||||
2025-11-27 22:48:07.375 | INFO | __main__:create_status_bar:161 - 创建状态栏
|
||||
2025-11-27 22:48:07.377 | INFO | __main__:create_menubar:168 - 创建菜单栏
|
||||
2025-11-27 22:48:07.397 | INFO | __main__:init_ui:60 - 界面初始化完成
|
||||
2025-11-27 22:48:07.565 | INFO | __main__:main:426 - 应用程序启动完成
|
||||
2025-11-27 22:48:08.529 | INFO | __main__:open_database:184 - 打开数据库文件对话框
|
||||
2025-11-27 22:48:10.594 | INFO | __main__:open_database:198 - 打开数据库文件: C:/Users/xiaji/Documents/个人文件夹/夏骥/hothub的抓取/product/products.db
|
||||
2025-11-27 22:48:10.595 | INFO | __main__:connect_to_database:208 - 数据库连接成功
|
||||
2025-11-27 22:48:10.596 | INFO | __main__:load_table_list:236 - 加载了 3 个表
|
||||
2025-11-27 22:48:12.872 | INFO | __main__:on_table_selected:246 - 选中表: product_analysis
|
||||
2025-11-27 22:48:12.882 | INFO | __main__:load_table_data:282 - 加载表 product_analysis 数据完成，共 251 行
|
||||
2025-11-27 22:48:12.883 | INFO | __main__:update_field_combo:312 - 更新字段下拉框: product_analysis, 共 9 个字段
|
||||
2025-11-27 22:49:47.902 | INFO | __main__:apply_filter:365 - 应用筛选条件: difficulty_score LIKE '%<75%', 匹配到 0 行数据
|
||||
2025-11-27 22:50:04.651 | INFO | __main__:apply_filter:365 - 应用筛选条件: difficulty_score LIKE '%difficulty_score<75%', 匹配到 0 行数据
|
||||
2025-11-27 22:50:44.808 | INFO | __main__:closeEvent:404 - 关闭应用程序
|
||||
2025-11-27 22:53:01.583 | INFO | __main__:__init__:27 - 初始化SQLite数据库查看器
|
||||
2025-11-27 22:53:01.583 | INFO | __main__:init_ui:34 - 设置主窗口界面
|
||||
2025-11-27 22:53:01.583 | INFO | __main__:create_top_buttons:64 - 创建顶部按钮
|
||||
2025-11-27 22:53:01.584 | INFO | __main__:create_filter_section:87 - 创建筛选控件区域
|
||||
2025-11-27 22:53:01.590 | INFO | __main__:create_splitter:132 - 创建分割器界面
|
||||
2025-11-27 22:53:01.591 | INFO | __main__:create_status_bar:161 - 创建状态栏
|
||||
2025-11-27 22:53:01.591 | INFO | __main__:create_menubar:168 - 创建菜单栏
|
||||
2025-11-27 22:53:01.600 | INFO | __main__:init_ui:60 - 界面初始化完成
|
||||
2025-11-27 22:53:01.727 | INFO | __main__:main:440 - 应用程序启动完成
|
||||
2025-11-27 22:53:03.101 | INFO | __main__:open_database:184 - 打开数据库文件对话框
|
||||
2025-11-27 22:53:04.822 | INFO | __main__:open_database:198 - 打开数据库文件: C:/Users/xiaji/Documents/个人文件夹/夏骥/hothub的抓取/product/products.db
|
||||
2025-11-27 22:53:04.823 | INFO | __main__:connect_to_database:208 - 数据库连接成功
|
||||
2025-11-27 22:53:04.824 | INFO | __main__:load_table_list:236 - 加载了 3 个表
|
||||
2025-11-27 22:53:42.968 | INFO | __main__:closeEvent:418 - 关闭应用程序
|
||||
@@ -1,11 +0,0 @@
|
||||
=== Product Hunt 产品信息 ===
|
||||
|
||||
产品名称: NoSho.app
|
||||
|
||||
产品简介: One simple link for businesses to grow their waitlist and fill last-minute availability with deposits fast. Make the availability you want booked fast visible to customers with one click, stop promoting slots you have hidden in a booking system maze. No more chasing messages or posting Stories that vanish in 24 hours. Just share your NoSho profile, automatically notify customers when new slots are added and get booked securely.
|
||||
|
||||
制作人发言: 未获取
|
||||
|
||||
用户数: 60 followers
|
||||
|
||||
提取时间: 2025-11-27 20:18:48
|
||||
Reference in New Issue
Block a user