admin/Nexus
1
0
Fork 0
Code Issues Pull Requests Actions 91 Packages Projects Releases Wiki Activity

feat(servers): 主列表宝塔登录 + 健康检查写心跳

Browse Source 
主服务器列表操作列一键登录宝塔；SSH 健康检查写入 Redis 心跳并刷新在线状态。

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
r
2026-06-21 11:05:43 +08:00
parent 2e797f86b7
commit 6720ca3474
13 changed files with 477 additions and 115 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/changelog/2026-06-21-health-check-write-heartbeat.md
+31
View File
@@ -0,0 +1,31 @@
# 健康检查写入 Redis 心跳
**日期**：2026-06-21
## 变更摘要
批量「健康检查」在 SSH 探测后，将结果写入 `heartbeat:{server_id}`（与 Agent 心跳同结构），列表/宝塔模块可立即反映在线状态；任务结果 stdout 附带心跳时间。
## 动机
健康检查仅返回成功/失败，不更新 Redis，导致 `#/servers` 状态列与宝塔列表仍显示旧心跳或 unknown。
## 涉及文件
- `server/application/server_connectivity.py` — `write_ssh_health_heartbeat`、`normalize_ssh_probe_system_info`
- `server/application/services/server_batch_service.py` — `_run_health_check` 写心跳
- `server/application/services/server_service.py` — `check_all_servers` 同步写心跳
- `server/api/servers.py` — `has_heartbeat` 时无 Agent 也显示 offline/online
- `tests/test_server_connectivity.py`、`tests/test_server_status_field.py`
## 迁移 / 重启
需重启 API worker（后端部署）。
## 验证方式
```bash
.venv/bin/pytest tests/test_server_connectivity.py tests/test_server_status_field.py -q
```
生产：对离线机执行健康检查后刷新 `#/servers`，状态列应更新；批量任务详情 stdout 含「心跳 2026-…」。
docs/changelog/2026-06-21-servers-page-btpanel-login.md
+29
View File
@@ -0,0 +1,29 @@
# 主服务器列表增加宝塔一键登录
**日期**：2026-06-21
## 变更摘要
在主服务器列表（`#/servers`）每行操作列增加「宝塔」按钮，调用 `POST /api/btpanel/servers/{id}/login-url` 在新标签页打开宝塔临时登录链接（API 优先，失败则 SSH 回退）。
## 动机
运维在浏览全量服务器时无需先进入宝塔子模块即可一键登录面板。
## 涉及文件
- `frontend/src/components/servers/ServerTableRowActions.vue` — 新增「宝塔」按钮
- `frontend/src/composables/btpanel/useBtPanelLogin.ts` — 登录逻辑复用
- `frontend/src/pages/ServersPage.vue` — 主表与未设路径表接线
- `frontend/src/components/servers/ServerUnsetPathPanel.vue` — 透传 loading / emit
- `frontend/src/constants/serverTableHeaders.ts` — 操作列宽 320→380
## 迁移 / 重启
无。仅前端构建部署。
## 验证方式
1. `cd frontend && npx vite build`
2. 打开 `#/servers`，任一行点击「宝塔」，应新标签打开宝塔登录页
3. 未配置 API 的机器仍可走 SSH 回退（需 SSH 可达且已装宝塔）
frontend/src/components/servers/ServerTableRowActions.vue
+129
View File
@@ -0,0 +1,129 @@
<template>
<div class="server-row-actions d-flex ga-1 align-center justify-end flex-nowrap">
<v-btn
:icon="pinnedSlot != null ? 'mdi-check' : 'mdi-plus'"
size="x-small"
variant="text"
:color="pinnedSlot != null ? 'success' : 'primary'"
density="compact"
class="server-row-actions__pin"
:title="pinnedSlot != null ? `已在监测 · 槽 ${pinnedSlot + 1}` : '加入实时监测'"
:loading="pinLoading"
@click.stop="$emit('pin')"
/>
<v-btn
variant="text"
size="x-small"
color="primary"
density="compact"
class="server-row-actions__btn"
@click.stop="$emit('terminal')"
>
终端
</v-btn>
<v-btn
variant="text"
size="x-small"
color="deep-orange"
density="compact"
class="server-row-actions__btn"
title="一键登录宝塔面板"
:loading="btLoginLoading"
@click.stop="$emit('bt-login')"
>
宝塔
</v-btn>
<v-btn
variant="text"
size="x-small"
density="compact"
class="server-row-actions__btn"
:disabled="!siteUrl"
:title="siteUrl ? `打开 ${siteHost || siteUrl}` : '无站点 URL'"
@click.stop="siteUrl && $emit('site')"
>
站点
</v-btn>
<v-btn
variant="text"
size="x-small"
density="compact"
class="server-row-actions__btn"
@click.stop="$emit('files')"
>
文件
</v-btn>
<v-btn
variant="text"
size="x-small"
density="compact"
class="server-row-actions__btn"
@click.stop="$emit('detect-path')"
>
检测路径
</v-btn>
<v-btn
variant="text"
size="x-small"
density="compact"
class="server-row-actions__btn"
:loading="diagnoseLoading"
@click.stop="$emit('diagnose')"
>
诊断
</v-btn>
<v-btn
variant="text"
size="x-small"
density="compact"
class="server-row-actions__btn"
@click.stop="$emit('edit')"
>
编辑
</v-btn>
<v-btn
variant="text"
size="x-small"
color="error"
density="compact"
class="server-row-actions__btn"
@click.stop="$emit('delete')"
>
删除
</v-btn>
</div>
</template>
<script setup lang="ts">
defineProps<{
pinnedSlot?: number | null
pinLoading?: boolean
btLoginLoading?: boolean
siteUrl?: string | null
siteHost?: string | null
diagnoseLoading?: boolean
}>()
defineEmits<{
pin: []
terminal: []
'bt-login': []
site: []
files: []
'detect-path': []
diagnose: []
edit: []
delete: []
}>()
</script>
<style scoped>
.server-row-actions__pin {
flex: 0 0 28px;
}
.server-row-actions__btn {
min-width: 52px;
padding-inline: 6px;
}
</style>
frontend/src/components/servers/ServerUnsetPathPanel.vue
+50 -35
View File
@@ -14,18 +14,20 @@
</v-chip>
<v-spacer />
<v-btn
size="small"
variant="tonal"
:disabled="total === 0"
:loading="selectAllLoading"
@click="$emit('select-all-filtered')"
prepend-icon="mdi-folder-search"
size="small"
:disabled="selectedCount === 0"
title="请先勾选至少一台服务器"
@click="$emit('detect-path')"
>
全选筛选结果 ({{ total }})
检测路径
</v-btn>
<v-btn
variant="text"
size="small"
prepend-icon="mdi-refresh"
class="ml-2"
:loading="loading"
@click="$emit('refresh')"
>
@@ -33,6 +35,21 @@
</v-btn>
</v-card-title>
<v-card-text class="pt-0 pb-2">
<div class="d-flex align-center flex-wrap ga-2">
<v-spacer />
<v-btn
size="small"
variant="tonal"
:disabled="total === 0"
:loading="selectAllLoading"
@click="$emit('select-all-filtered')"
>
全选筛选结果 ({{ total }})
</v-btn>
</div>
</v-card-text>
<ServerBatchActionBar
:count="selectedCount"
@batch-category="$emit('batch-category')"
@@ -149,26 +166,23 @@
</template>
<template #item.actions="{ item }">
<div class="d-flex ga-1">
<v-btn variant="text" size="x-small" color="primary" density="compact" @click.stop="$emit('terminal', item)">
终端
</v-btn>
<v-btn variant="text" size="x-small" density="compact" @click.stop="$emit('files', item)">
文件
</v-btn>
<v-btn variant="text" size="x-small" density="compact" @click.stop="$emit('detect-path-single', item)">
检测路径
</v-btn>
<v-btn variant="text" size="x-small" density="compact" @click.stop="$emit('agent-diagnose', item)">
诊断
</v-btn>
<v-btn variant="text" size="x-small" density="compact" @click.stop="$emit('edit', item)">
编辑
</v-btn>
<v-btn variant="text" size="x-small" color="error" density="compact" @click.stop="$emit('delete', item)">
删除
</v-btn>
</div>
<ServerTableRowActions
:pinned-slot="pinnedMap[item.id] ?? null"
:pin-loading="pinLoadingId === item.id"
:bt-login-loading="btLoginLoadingId === item.id"
:site-url="resolveServerSiteUrl(item)"
:site-host="resolveServerSiteHost(item)"
:diagnose-loading="agentDiagnoseLoadingId === item.id"
@pin="$emit('pin', item)"
@terminal="$emit('terminal', item)"
@bt-login="$emit('bt-login', item)"
@site="$emit('site', item)"
@files="$emit('files', item)"
@detect-path="$emit('detect-path-single', item)"
@diagnose="$emit('agent-diagnose', item)"
@edit="$emit('edit', item)"
@delete="$emit('delete', item)"
/>
</template>
<template #expanded-row="{ columns, item }">
@@ -198,6 +212,9 @@ import { statusChipColor, statusLabel, formatRelativeTime } from '@/utils/status
import { normalizeServerIds } from '@/utils/serverSelection'
import ServerInlineDetail from '@/components/servers/ServerInlineDetail.vue'
import ServerBatchActionBar from '@/components/servers/ServerBatchActionBar.vue'
import ServerTableRowActions from '@/components/servers/ServerTableRowActions.vue'
import { SERVER_DATA_TABLE_HEADERS } from '@/constants/serverTableHeaders'
import { resolveServerSiteHost, resolveServerSiteUrl } from '@/utils/serverSiteUrl'
const props = defineProps<{
highlight?: boolean
@@ -214,6 +231,10 @@ const props = defineProps<{
editingPathId: number | null
editingPathValue: string
savingPathId: number | null
pinnedMap: Record<number, number>
pinLoadingId: number | null
btLoginLoadingId: number | null
agentDiagnoseLoadingId: number | null
}>()
const emit = defineEmits<{
@@ -227,7 +248,10 @@ const emit = defineEmits<{
'select-all-filtered': []
'toggle-expand': [item: ServerApiItem]
terminal: [item: ServerApiItem]
'bt-login': [item: ServerApiItem]
files: [item: ServerApiItem]
site: [item: ServerApiItem]
pin: [item: ServerApiItem]
'detect-path-single': [item: ServerApiItem]
'agent-diagnose': [item: ServerApiItem]
edit: [item: ServerApiItem]
@@ -245,16 +269,7 @@ const emit = defineEmits<{
'clear-selection': []
}>()
const headers = [
{ title: '状态', key: 'status', width: 100 },
{ title: '名称', key: 'name' },
{ title: '地址', key: 'domain' },
{ title: '分类', key: 'category', width: 100 },
{ title: '目标路径', key: 'target_path', width: 200 },
{ title: 'Agent', key: 'agent_version', width: 100 },
{ title: '心跳', key: 'last_heartbeat', width: 120 },
{ title: '操作', key: 'actions', width: 260, align: 'end' as const, sortable: false },
]
const headers = SERVER_DATA_TABLE_HEADERS
const selectedCount = computed(() => normalizeServerIds(props.selectedItems).length)
frontend/src/composables/btpanel/useBtPanelLogin.ts
+21
View File
@@ -0,0 +1,21 @@
import { ref } from 'vue'
import { createBtLoginUrl } from '@/api/btpanel'
export function useBtPanelLogin() {
const loadingId = ref<number | null>(null)
async function openBtLogin(serverId: number) {
loadingId.value = serverId
try {
const { url } = await createBtLoginUrl(serverId)
window.open(url, '_blank', 'noopener,noreferrer')
window.$snackbar?.('已打开宝塔登录链接', 'success')
} catch (e: unknown) {
window.$snackbar?.(e instanceof Error ? e.message : '生成宝塔登录链接失败', 'error')
} finally {
loadingId.value = null
}
}
return { loadingId, openBtLogin }
}
frontend/src/constants/serverTableHeaders.ts
+11
View File
@@ -0,0 +1,11 @@
/** Shared column config for main / unset-path server tables. */
export const SERVER_DATA_TABLE_HEADERS = [
{ title: '状态', key: 'status', width: 100 },
{ title: '名称', key: 'name' },
{ title: '地址', key: 'domain' },
{ title: '分类', key: 'category', width: 100 },
{ title: '目标路径', key: 'target_path', width: 200 },
{ title: 'Agent', key: 'agent_version', width: 100 },
{ title: '心跳', key: 'last_heartbeat', width: 120 },
{ title: '操作', key: 'actions', width: 380, align: 'end' as const, sortable: false },
]
frontend/src/pages/ServersPage.vue
+70 -77
View File
@@ -34,6 +34,54 @@
<span class="text-caption text-medium-emphasis">点击顶部统计卡可切换筛选或定位</span>
</div>
<ServerUnsetPathPanel
:highlight="unsetPathFocus"
:servers="unsetPathServers"
:loading="unsetPathLoading"
:total="unsetPathTotal"
:page="unsetPathPage"
:items-per-page="unsetPathItemsPerPage"
:items-per-page-options="dataTablePageOptions"
:expanded-ids="expandedUnsetIds"
v-model:selected-items="unsetSelectedItems"
v-model:sort-by="sortBy"
:select-all-loading="selectAllUnsetLoading"
:editing-path-id="editingPathId"
:editing-path-value="editingPathValue"
:saving-path-id="savingPathId"
:pinned-map="pinnedMap"
:pin-loading-id="pinLoadingId"
:bt-login-loading-id="btLoginLoadingId"
:agent-diagnose-loading-id="agentDiagnoseLoadingId"
@refresh="loadUnsetPathServers()"
@update:page="onUnsetPageChange"
@update:items-per-page="onUnsetItemsPerPageChange"
@update:expanded-ids="expandedUnsetIds = $event"
@update:editing-path-value="editingPathValue = $event"
@select-all-filtered="selectAllUnsetFiltered"
@toggle-expand="toggleUnsetExpand"
@terminal="openTerminal"
@bt-login="openBtLogin($event.id)"
@files="openFiles"
@site="openBrowser"
@pin="onPinServer"
@detect-path-single="detectPathSingle"
@agent-diagnose="openAgentDiagnose"
@edit="editServer"
@delete="confirmDelete"
@edit-path="startEditPath"
@save-path="saveTargetPath"
@cancel-edit-path="cancelEditPath"
@batch-category="openBatchCategory('unset')"
@health-check="batchHealthCheck('unset')"
@detect-path="openDetectPathConfirm('unset')"
@install-agent="batchInstallAgent('unset')"
@upgrade-agent="batchUpgradeAgent('unset')"
@uninstall-agent="batchUninstallAgent('unset')"
@batch-delete="confirmBatchDelete('unset')"
@clear-selection="unsetSelectedItems = []"
/>
<!-- Server Table -->
<v-card elevation="0" rounded="lg" class="my-5" border>
<v-card-title class="d-flex align-center">
@@ -279,32 +327,23 @@
</template>
<template #item.actions="{ item }">
<div class="d-flex ga-1 align-center">
<v-btn
:icon="pinnedMap[item.id] != null ? 'mdi-check' : 'mdi-plus'"
size="x-small"
variant="text"
:color="pinnedMap[item.id] != null ? 'success' : 'primary'"
density="compact"
:title="pinnedMap[item.id] != null ? `已在监测 · 槽 ${pinnedMap[item.id]! + 1}` : '加入实时监测'"
:loading="pinLoadingId === item.id"
@click.stop="onPinServer(item)"
/>
<v-btn variant="text" size="x-small" color="primary" density="compact" @click.stop="openTerminal(item)">终端</v-btn>
<v-btn
v-if="resolveServerSiteUrl(item)"
variant="text"
size="x-small"
density="compact"
@click.stop="openBrowser(item)"
>
站点
</v-btn>
<v-btn variant="text" size="x-small" density="compact" @click.stop="openFiles(item)">文件</v-btn>
<v-btn variant="text" size="x-small" density="compact" :loading="agentDiagnoseLoadingId === item.id" @click.stop="openAgentDiagnose(item)">诊断</v-btn>
<v-btn variant="text" size="x-small" density="compact" @click.stop="editServer(item)">编辑</v-btn>
<v-btn variant="text" size="x-small" color="error" density="compact" @click.stop="confirmDelete(item)">删除</v-btn>
</div>
<ServerTableRowActions
:pinned-slot="pinnedMap[item.id] ?? null"
:pin-loading="pinLoadingId === item.id"
:bt-login-loading="btLoginLoadingId === item.id"
:site-url="resolveServerSiteUrl(item)"
:site-host="resolveServerSiteHost(item)"
:diagnose-loading="agentDiagnoseLoadingId === item.id"
@pin="onPinServer(item)"
@terminal="openTerminal(item)"
@bt-login="openBtLogin(item.id)"
@site="openBrowser(item)"
@files="openFiles(item)"
@detect-path="detectPathSingle(item)"
@diagnose="openAgentDiagnose(item)"
@edit="editServer(item)"
@delete="confirmDelete(item)"
/>
</template>
<template #expanded-row="{ columns, item }">
@@ -411,47 +450,6 @@
</div>
</v-card>
<ServerUnsetPathPanel
:highlight="unsetPathFocus"
:servers="unsetPathServers"
:loading="unsetPathLoading"
:total="unsetPathTotal"
:page="unsetPathPage"
:items-per-page="unsetPathItemsPerPage"
:items-per-page-options="dataTablePageOptions"
:expanded-ids="expandedUnsetIds"
v-model:selected-items="unsetSelectedItems"
v-model:sort-by="sortBy"
:select-all-loading="selectAllUnsetLoading"
:editing-path-id="editingPathId"
:editing-path-value="editingPathValue"
:saving-path-id="savingPathId"
@refresh="loadUnsetPathServers()"
@update:page="onUnsetPageChange"
@update:items-per-page="onUnsetItemsPerPageChange"
@update:expanded-ids="expandedUnsetIds = $event"
@update:editing-path-value="editingPathValue = $event"
@select-all-filtered="selectAllUnsetFiltered"
@toggle-expand="toggleUnsetExpand"
@terminal="openTerminal"
@files="openFiles"
@detect-path-single="detectPathSingle"
@agent-diagnose="openAgentDiagnose"
@edit="editServer"
@delete="confirmDelete"
@edit-path="startEditPath"
@save-path="saveTargetPath"
@cancel-edit-path="cancelEditPath"
@batch-category="openBatchCategory('unset')"
@health-check="batchHealthCheck('unset')"
@detect-path="openDetectPathConfirm('unset')"
@install-agent="batchInstallAgent('unset')"
@upgrade-agent="batchUpgradeAgent('unset')"
@uninstall-agent="batchUninstallAgent('unset')"
@batch-delete="confirmBatchDelete('unset')"
@clear-selection="unsetSelectedItems = []"
/>
<!-- Pending / failed SSH connections -->
<v-card id="pending-servers-panel" elevation="0" rounded="lg" class="my-5" border>
<v-card-title class="d-flex align-center">
@@ -770,12 +768,15 @@ import WatchSlotRow from '@/components/watch/WatchSlotRow.vue'
import { useWatchPins } from '@/composables/useWatchPins'
import ServerFormDialog from '@/components/servers/ServerFormDialog.vue'
import ServerBatchActionBar from '@/components/servers/ServerBatchActionBar.vue'
import ServerTableRowActions from '@/components/servers/ServerTableRowActions.vue'
import ServerInlineDetail from '@/components/servers/ServerInlineDetail.vue'
import ServerUnsetPathPanel from '@/components/servers/ServerUnsetPathPanel.vue'
import { SERVER_DATA_TABLE_HEADERS } from '@/constants/serverTableHeaders'
import CredentialsDialog from '@/components/credentials/CredentialsDialog.vue'
import AgentDiagnoseDialog from '@/components/servers/AgentDiagnoseDialog.vue'
import type { AgentDiagnoseResult } from '@/types/agentDiagnose'
import { useServerFormDialog } from '@/composables/servers/useServerFormDialog'
import { useBtPanelLogin } from '@/composables/btpanel/useBtPanelLogin'
import { DATA_TABLE_ITEMS_PER_PAGE_OPTIONS } from '@/constants/dataTable'
import {
SERVER_STAT_CARD_KEYS,
@@ -856,6 +857,7 @@ const dataTablePageOptions = [...DATA_TABLE_ITEMS_PER_PAGE_OPTIONS]
const snackbar = useSnackbar()
const { pinnedMap, pinServer: doPinServer, slots: watchSlots } = useWatchPins()
const pinLoadingId = ref<number | null>(null)
const { loadingId: btLoginLoadingId, openBtLogin } = useBtPanelLogin()
const showReplacePin = ref(false)
const replacePinServerId = ref<number | null>(null)
@@ -1415,16 +1417,7 @@ const selectedItems = ref<Array<ServerApiItem | number>>([])
const selectedIds = computed(() => new Set(normalizeServerIds(selectedItems.value)))
// ── Table config ──
const headers = [
{ title: '状态', key: 'status', width: 100 },
{ title: '名称', key: 'name' },
{ title: '地址', key: 'domain' },
{ title: '分类', key: 'category', width: 100 },
{ title: '目标路径', key: 'target_path', width: 200 },
{ title: 'Agent', key: 'agent_version', width: 100 },
{ title: '心跳', key: 'last_heartbeat', width: 120 },
{ title: '操作', key: 'actions', width: 200, align: 'end' as const, sortable: false },
]
const headers = SERVER_DATA_TABLE_HEADERS
const statsBooting = ref(true)
const statsRefreshing = ref(false)
server/api/servers.py
+9 -2
View File
@@ -2138,10 +2138,16 @@ async def _build_server_list_with_overlay(redis, servers: list) -> list[dict]:
return result
def _derive_server_status(is_online: bool | None, agent_version: str | None, *, has_agent: bool = False) -> str:
def _derive_server_status(
is_online: bool | None,
agent_version: str | None,
*,
has_agent: bool = False,
has_heartbeat: bool = False,
) -> str:
"""Map live/DB fields to frontend status: online | offline | unknown."""
has_agent = has_agent or bool(agent_version and str(agent_version).strip())
if has_agent or is_online:
if has_agent or has_heartbeat or is_online:
return "online" if is_online else "offline"
return "unknown"
@@ -2177,6 +2183,7 @@ def _apply_heartbeat_overlay(server_data: dict, server: Server, heartbeat: dict
server_data.get("is_online"),
server_data.get("agent_version") or server.agent_version,
has_agent=installed,
has_heartbeat=bool(heartbeat),
)
_apply_agent_guidance(server_data, server, heartbeat)
server/application/server_connectivity.py
+46
View File
@@ -2,6 +2,9 @@
from __future__ import annotations
import json
import logging
from datetime import datetime, timezone
from typing import Any
from sqlalchemy import and_, or_, select
@@ -9,6 +12,8 @@ from sqlalchemy import and_, or_, select
from server.domain.models import Server
from server.infrastructure.database.server_repo import _target_path_unset_filter
logger = logging.getLogger("nexus.server_connectivity")
REDIS_KEY_PREFIX = "heartbeat:"
@@ -102,3 +107,44 @@ async def scan_monitored_connectivity(
"offline": offline,
"offline_servers": offline_servers,
}
def normalize_ssh_probe_system_info(probe: dict[str, Any]) -> dict[str, Any]:
"""Flatten ssh_health_probe payload into Agent-compatible system_info."""
raw = probe.get("system_info") or {}
if isinstance(raw, dict) and isinstance(raw.get("system_info"), dict):
system_info = dict(raw["system_info"])
elif isinstance(raw, dict):
system_info = dict(raw)
else:
system_info = {}
system_info["probe_channel"] = str(probe.get("channel") or "ssh")
if probe.get("status") != "online" and probe.get("error"):
system_info["probe_error"] = str(probe["error"])[:300]
return system_info
async def write_ssh_health_heartbeat(
redis: Any,
server: Server,
probe: dict[str, Any],
) -> str:
"""Write SSH health-check result to Redis heartbeat (same shape as Agent). Returns last_heartbeat ISO."""
from server.background.heartbeat_flush import FLUSH_INTERVAL
is_online = probe.get("status") == "online"
system_info = normalize_ssh_probe_system_info(probe)
now = datetime.now(timezone.utc).isoformat()
agent_version = (server.agent_version or "").strip()
key = f"{REDIS_KEY_PREFIX}{server.id}"
await redis.hset(key, mapping={
"is_online": str(is_online),
"system_info": json.dumps(system_info, ensure_ascii=False),
"last_heartbeat": now,
"agent_version": agent_version,
"time_drift_seconds": "0",
"drift_level": "ok",
})
await redis.expire(key, int(FLUSH_INTERVAL * 1.5))
return now
server/application/services/server_batch_service.py
+12 -1
View File
@@ -476,6 +476,7 @@ async def _run_category(live: dict[str, Any]) -> None:
async def _run_health_check(live: dict[str, Any]) -> None:
from server.application.server_connectivity import write_ssh_health_heartbeat
from server.infrastructure.ssh.remote_probe import ssh_health_probe
async def handler(sid: int, server_map: dict[int, Server], labels: dict[int, str], _live) -> dict:
@@ -487,11 +488,21 @@ async def _run_health_check(live: dict[str, Any]) -> None:
probe = await ssh_health_probe(server, timeout=15)
online = probe.get("status") == "online"
detail = probe.get("error") or probe.get("channel") or probe.get("status") or ""
last_hb = ""
try:
last_hb = await write_ssh_health_heartbeat(get_redis(), server, probe)
except Exception as redis_err:
logger.warning("health check heartbeat write failed server=%s: %s", sid, redis_err)
hb_note = f" · 心跳 {last_hb[:19]}" if last_hb else ""
status_label = "在线" if online else "离线"
line = f"{status_label}{hb_note}"
if detail:
line = f"{line} · {detail}"
return result_item_dict(
server_id=sid,
server_name=label,
success=online,
stdout=str(detail)[:500] if online else "",
stdout=line[:500],
error="" if online else str(probe.get("error") or "离线")[:300],
)
except Exception as e:
server/application/services/server_service.py
+12
View File
@@ -41,13 +41,25 @@ class ServerService:
async def check_all_servers(self, server_ids: List[int]) -> Dict[int, dict]:
"""Check health of multiple servers via SSH (no Agent port 8601 required)."""
from server.application.server_connectivity import write_ssh_health_heartbeat
from server.infrastructure.redis.client import get_redis
from server.infrastructure.ssh.remote_probe import ssh_health_probe
results = {}
redis = get_redis()
async def _check_one(server: Server) -> tuple:
try:
probe = await ssh_health_probe(server, timeout=15)
try:
last_hb = await write_ssh_health_heartbeat(redis, server, probe)
probe = {**probe, "last_heartbeat": last_hb}
except Exception as redis_err:
logger.warning(
"health check heartbeat write failed server=%s: %s",
server.id,
redis_err,
)
return server.id, probe
except Exception as e:
return server.id, {"status": "offline", "error": str(e)[:200], "channel": "ssh"}
tests/test_server_connectivity.py
+49
View File
@@ -9,6 +9,8 @@ import pytest
from server.application.server_connectivity import (
count_monitored_connectivity,
item_matches_online_filter,
normalize_ssh_probe_system_info,
write_ssh_health_heartbeat,
)
@@ -42,3 +44,50 @@ async def test_count_monitored_connectivity_batches_redis():
out = await count_monitored_connectivity(redis, session, batch_size=2)
assert out == {"monitored": 3, "online": 1, "offline": 2}
def test_normalize_ssh_probe_system_info_nested():
probe = {
"status": "online",
"channel": "ssh",
"system_info": {
"status": "healthy",
"system_info": {"cpu_usage": 12.5, "probe": "ssh"},
},
}
out = normalize_ssh_probe_system_info(probe)
assert out["cpu_usage"] == 12.5
assert out["probe_channel"] == "ssh"
def test_normalize_ssh_probe_system_info_offline_error():
probe = {"status": "offline", "channel": "ssh", "error": "timeout"}
out = normalize_ssh_probe_system_info(probe)
assert out["probe_error"] == "timeout"
assert out["probe_channel"] == "ssh"
@pytest.mark.asyncio
async def test_write_ssh_health_heartbeat_redis_shape():
redis = AsyncMock()
server = MagicMock()
server.id = 42
server.agent_version = "2.1.0"
ts = await write_ssh_health_heartbeat(
redis,
server,
{
"status": "online",
"channel": "ssh",
"system_info": {"status": "healthy", "system_info": {"cpu_usage": 1.0}},
},
)
assert ts
redis.hset.assert_awaited_once()
mapping = redis.hset.await_args.kwargs["mapping"]
assert mapping["is_online"] == "True"
assert mapping["agent_version"] == "2.1.0"
assert "cpu_usage" in mapping["system_info"]
redis.expire.assert_awaited_once()
tests/test_server_status_field.py
+8
View File
@@ -17,3 +17,11 @@ def test_unknown_without_agent():
def test_offline_with_agent_key_only():
assert _derive_server_status(False, None, has_agent=True) == "offline"
def test_offline_with_ssh_heartbeat_only():
assert _derive_server_status(False, None, has_heartbeat=True) == "offline"
def test_online_with_ssh_heartbeat_only():
assert _derive_server_status(True, None, has_heartbeat=True) == "online"