xiaji
|
5eb331bf2d
|
fix(api_push): naive published_at 改当 Asia/Shanghai,不再当 UTC
症状: 财联社/微信等中国源推 ingest 时,payload published_at 通常不带 tz
(naive 字符串),含义是北京时间的墙上时间。
后端之前用 .replace(tzinfo=timezone.utc) 强行当 UTC,导致入库后:
'2026-06-15 19:58:42' (naive,真意=北京 19:58) →
PG 存 2026-06-15 19:58:42+00 = 渲染成 2026-06-16 03:58:42+08
前端 dayjs(...).fromNow() 算到当前 +08 时间 20:55 → 显示 '7 小时内'(未来时间)
根因: 中国源给的 naive 字符串实际就是 Asia/Shanghai,不该被当 UTC
修法:
- 新增 zoneinfo + settings.tz 依赖
- naive → settings.tz(默认 Asia/Shanghai),再 astimezone(UTC) 入库
- aware → astimezone(UTC) 归一
注意: 只修未来的新数据;已入库的错位数据需要 backfill(见 PR 后讨论)
|
2026-06-15 21:01:45 +08:00 |
|
xiaji
|
07534eb144
|
feat(ingest): API Push 短新闻接口层
- POST /api/v1/ingest:鉴权(X-Ingest-Token) + 限速(每 token 2 篇/秒,
Redis 滑动桶,INGEST_RATE_PER_SEC 可调) + 三层去重(L1 external_id /
L2 content_hash / L3 DB UNIQUE 兜底,均带 reason)
- 写入字段:is_short_news=True、translation/format/image_ai_status='n/a'、
classify_status=(有 tags?'ok':'pending')、commentary_{angel,meituan}_status='pending'、
body_zh_text=body_text(走统一路径,前端/prompt 不用改)
- services/fetchers/api_push.py:compute_content_hash + synthesize_url +
normalize_published_at + build_initial_status 纯函数
- schemas/ingest.py:IngestPayload(title 1-200/body 1-5000/tags 去重去空) +
IngestResponse(article_id/content_hash/status/reason/matched_external_id)
- admin.py:POST/GET/DELETE /admin/sources/{id}/ingest-tokens — owner 生成
(raw_token 仅一次性返回)、列出、撤销
- schemas/article.py:ArticleListItem 加 is_short_news/source_ref;
ArticleDetail 加 is_short_news/source_ref/external_id
- main.py:挂 ingest router;config.py + .env.example:ingest_rate_per_sec 默认 2
短新闻由 commit 1 enrichment_loop 自动接管 classify + 双 provider commentary,
跳过 format/image。
|
2026-06-14 16:04:45 +08:00 |
|
Mavis
|
a5548d6e64
|
fix(fetcher): fulltext 抓取用真实浏览器 UA,绕过 NHK 等 403
|
2026-06-08 15:55:30 +08:00 |
|
Mavis
|
9862a92423
|
perf: 翻译独立后台循环(1 篇/秒)+ Semaphore 1
之前 fetch_one_source 入库后立即调翻译(可能并发触发腾讯 TMT 限速)
改为独立 translation_loop 后台循环:
- 完全不和 RSS 抓取并行
- 1 篇/秒节拍(Semaphore 1 + sleep 1.0)
- 没活时空闲 5 秒再轮询
- pending/failed 都重试
|
2026-06-08 00:27:09 +08:00 |
|
Mavis
|
60b062daf2
|
feat: initial MVP - FastAPI backend + Vue3 frontend + docker-compose
- backend: FastAPI + SQLAlchemy 2.0(async) + asyncpg + Alembic
- 7 API routes: auth/me/articles/sources/bookmarks/subscriptions/admin
- models: User/Source/Article/Bookmark/Subscription/ApiToken
- services: RSS fetcher (feedparser) + Tencent TMT translator with quota + cache + local NLLB fallback
- workers: APScheduler + asyncio pipeline (fetch -> dedupe -> insert -> translate)
- seed scripts: create_user, seed_sources (5 RSS: Reuters/BBC/Al Jazeera/NHK/DW)
- frontend: Vue 3 + Vite + Naive UI + Pinia + vue-router
- pages: Login, Feed (24h), ArticleDetail, Sources, Bookmarks, AdminSources
- deploy: docker-compose (postgres/redis/api/worker/frontend/caddy)
- docs: README, DEPLOY, architecture, acceptance
|
2026-06-07 21:51:01 +08:00 |
|