AirGrip
本文最后更新于 2025-12-25，文章内容可能已经过时。
Vision Hand Pro (AirGrip) 使用说明文档
Vision Hand Pro 是一个基于 Web 前端技术的隔空手势交互演示项目。它利用 TensorFlow.js 和 MediaPipe Hands 技术，无需任何额外硬件，仅通过普通的电脑摄像头即可实现高精度的手势识别与交互。
🚀 快速开始
打开文件：直接在浏览器（推荐 Chrome 或 Edge）中打开
AirGrip.html文件。授予权限：浏览器会提示请求摄像头权限，请点击“允许”。
初始化：等待页面加载模型（首次加载可能需要几秒钟），出现 "Initializing Neural Engine..." 提示。
开始交互：当看到 "Place Hand in View" 提示时，将一只手举起放入摄像头视野内即可。
🖐️ 基本操作
1. 光标控制 (Cursor)
定位：屏幕上的虚拟光标会跟随你的食指指尖移动。
状态：
⚪️ 空心圆环：光标处于悬浮状态。
⚪️ 实心白点：识别到“捏合”动作（点击/按下状态）。
2. 手势交互 (Gestures)
点击 (Click)：将 拇指 与 食指 指尖捏合（Pinch）。
拖拽 (Drag)：捏合手指后保持捏合状态并移动手掌。
释放 (Release)：松开拇指与食指。
🛠️ 功能模式
屏幕下方悬浮的 Dock 栏提供了三种功能按钮，使用手势光标移动到按钮上并执行“捏合”动作即可触发。
🎨 绘图模式 (Paint)
进入方式：点击底部的 “绘图” 按钮。
操作：
在空中捏合手指（就像捏着一支笔）并移动，即可在屏幕上绘制线条。
松开手指停止绘制。
清除画布：点击底部的 “清除” 按钮可以清空所有画迹。
🎵 音乐模式 (Music)
进入方式：点击底部的 “音乐” 按钮。
界面：右侧会滑出音乐播放面板。
操作：
滚动列表：在播放列表区域捏合手指并上下拖动，即可滚动列表。
播放歌曲：将光标移动到歌曲条目上，执行捏合动作即可播放。
播放控制：点击面板底部的播放/暂停按钮。
🖱️ 普通模式
默认模式，主要用于演示手势追踪和基本的点击交互。
📊 界面说明 (HUD)
左上角仪表盘：
FPS：显示当前的渲染帧率。
Gesture：显示当前识别到的手势状态（如
None或Pinch）。
中间提示：显示当前所处的模式（Mode）。
⚠️ 常见问题与技巧
光标抖动严重？
尝试增加环境光线，确保手部轮廓清晰。
手掌尽量正对摄像头。
无法识别手势？
请确保摄像头画面中只有一只手。
距离摄像头保持 0.5 米至 1 米左右的适中距离。
加载卡住？
请检查网络连接，项目首次运行需要从 CDN 下载 AI 模型文件。
确保浏览器支持 WebGL。
作者: 芝麻 | 技术栈: TensorFlow.js, MediaPipe, HTML5 Canvas
代码
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no">
<title>Vision Hand Pro</title>
<style>
:root {
--bg-color: #000000;
--card-bg: #1C1C1E;
--text-primary: #FFFFFF;
--text-secondary: #86868B;
--accent-blue: #0A84FF;
--accent-green: #30D158;
--accent-pink: #FF375F;
--glass-bg: rgba(28, 28, 30, 0.65);
--glass-border: rgba(255, 255, 255, 0.1);
--shadow-sm: 0 4px 12px rgba(0, 0, 0, 0.2);
--radius-xl: 32px;
--radius-md: 18px;
}
* {
margin: 0;
padding: 0;
box-sizing: border-box;
-webkit-font-smoothing: antialiased;
-moz-osx-font-smoothing: grayscale;
user-select: none;
}
body {
background-color: var(--bg-color);
color: var(--text-primary);
font-family: -apple-system, BlinkMacSystemFont, "SF Pro Display", "SF Pro Text", "Helvetica Neue", Helvetica, Arial, sans-serif;
min-height: 100vh;
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
overflow: hidden;
}
/* Immersive Container */
.immersive-container {
position: relative;
width: 100vw;
height: 100vh;
overflow: hidden;
}
/* Video Layer - Smart Crop */
#video {
position: absolute;
top: 50%;
left: 50%;
transform: translate(-50%, -50%) scaleX(-1);
min-width: 100%;
min-height: 100%;
width: auto;
height: auto;
object-fit: cover;
opacity: 0;
transition: opacity 1s ease;
filter: brightness(0.7) contrast(1.1); /* Cinematic look */
}
#video.loaded {
opacity: 1;
}
/* Canvas Layer - High Performance */
#canvas, #drawingCanvas {
position: absolute;
top: 0;
left: 0;
width: 100%;
height: 100%;
pointer-events: none;
}
#canvas { z-index: 2; }
#drawingCanvas { z-index: 1; opacity: 0; transition: opacity 0.5s; }
#drawingCanvas.active { opacity: 1; }
/* UI Overlay Layer */
.ui-layer {
position: absolute;
top: 0;
left: 0;
width: 100%;
height: 100%;
pointer-events: none;
z-index: 10;
}
/* HUD Elements */
.hud-top-left {
position: absolute;
top: 40px;
left: 40px;
display: flex;
gap: 16px;
}
.hud-card {
background: var(--glass-bg);
backdrop-filter: blur(40px) saturate(180%);
-webkit-backdrop-filter: blur(40px) saturate(180%);
border: 1px solid var(--glass-border);
padding: 16px 24px;
border-radius: var(--radius-md);
display: flex;
align-items: center;
gap: 12px;
box-shadow: var(--shadow-sm);
transform: translateZ(0);
transition: transform 0.2s cubic-bezier(0.25, 0.46, 0.45, 0.94);
}
.hud-card.active {
background: rgba(48, 209, 88, 0.15);
border-color: rgba(48, 209, 88, 0.3);
transform: scale(1.05);
}
.hud-label {
font-size: 12px;
font-weight: 600;
text-transform: uppercase;
color: var(--text-secondary);
letter-spacing: 0.05em;
}
.hud-value {
font-size: 24px;
font-weight: 700;
font-variant-numeric: tabular-nums;
}
/* Virtual Cursor */
.cursor {
position: absolute;
width: 40px;
height: 40px;
border: 2px solid rgba(255, 255, 255, 0.8);
border-radius: 50%;
pointer-events: none;
transform: translate(-50%, -50%) scale(0);
transition: transform 0.1s cubic-bezier(0.16, 1, 0.3, 1), background-color 0.2s;
box-shadow: 0 0 20px rgba(255, 255, 255, 0.3);
z-index: 20;
mix-blend-mode: exclusion;
}
.cursor.visible {
transform: translate(-50%, -50%) scale(1);
}
.cursor.clicking {
transform: translate(-50%, -50%) scale(0.8);
background-color: rgba(255, 255, 255, 0.8);
}
/* Interactive Elements Demo */
.interactive-area {
position: absolute;
top: 40px; /* Moved to top */
left: 50%;
transform: translateX(-50%);
display: flex;
gap: 20px;
pointer-events: auto; /* Allow interaction logic */
background: rgba(20, 20, 20, 0.8);
backdrop-filter: blur(20px);
padding: 10px 20px;
border-radius: 999px;
border: 1px solid rgba(255, 255, 255, 0.1);
z-index: 100; /* Ensure on top */
}
.action-btn {
min-width: 80px;
height: 44px;
padding: 0 20px;
border-radius: 22px;
background: rgba(255, 255, 255, 0.1);
border: 1px solid rgba(255, 255, 255, 0.2);
backdrop-filter: blur(20px);
display: flex;
justify-content: center;
align-items: center;
font-size: 14px;
font-weight: 600;
transition: all 0.3s cubic-bezier(0.34, 1.56, 0.64, 1);
position: relative;
text-transform: uppercase;
letter-spacing: 0.5px;
}
.action-btn.active {
background: var(--accent-blue);
color: white;
border-color: var(--accent-blue);
box-shadow: 0 0 20px rgba(10, 132, 255, 0.4);
}
.action-btn.hovered {
transform: scale(1.1);
background: rgba(255, 255, 255, 0.2);
}
.action-btn.clicked {
transform: scale(0.95);
}
/* Copyright Footer */
.copyright-footer {
position: absolute;
bottom: 20px;
left: 50%;
transform: translateX(-50%);
color: rgba(255, 255, 255, 0.4);
font-size: 12px;
z-index: 100;
pointer-events: auto;
text-align: center;
white-space: nowrap;
}
.copyright-footer a {
color: inherit;
text-decoration: none;
transition: color 0.2s;
}
.copyright-footer a:hover {
color: var(--text-primary);
text-decoration: underline;
}
/* Music Player Panel */
#musicPanel {
position: absolute;
right: -400px;
top: 50%;
transform: translateY(-50%);
width: 320px;
height: 70vh;
background: rgba(30, 30, 30, 0.85);
backdrop-filter: blur(30px);
border-radius: 24px;
border: 1px solid rgba(255, 255, 255, 0.1);
transition: right 0.5s cubic-bezier(0.16, 1, 0.3, 1);
display: flex;
flex-direction: column;
overflow: hidden;
pointer-events: auto;
padding: 20px;
}
#musicPanel.visible {
right: 40px;
}
.music-header {
font-size: 18px;
font-weight: 700;
margin-bottom: 16px;
display: flex;
align-items: center;
gap: 10px;
}
.playlist {
flex: 1;
overflow-y: auto;
margin-bottom: 20px;
}
/* Hide scrollbar */
.playlist::-webkit-scrollbar { display: none; }
.track-item {
display: flex;
align-items: center;
padding: 12px;
border-radius: 12px;
gap: 12px;
transition: background 0.2s;
margin-bottom: 8px;
}
.track-item.hovered {
background: rgba(255, 255, 255, 0.1);
}
.track-item.active {
background: rgba(255, 255, 255, 0.2);
}
.track-info {
flex: 1;
min-width: 0;
}
.track-title {
font-size: 14px;
font-weight: 600;
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
.track-artist {
font-size: 12px;
color: var(--text-secondary);
}
.mini-player {
background: rgba(0, 0, 0, 0.3);
padding: 16px;
border-radius: 16px;
display: flex;
align-items: center;
gap: 12px;
}
.play-btn {
width: 40px;
height: 40px;
border-radius: 50%;
background: #fff;
color: #000;
display: flex;
align-items: center;
justify-content: center;
font-size: 18px;
}
/* Loader */
.loader-overlay {
position: absolute;
top: 0; left: 0; right: 0; bottom: 0;
background: #000;
display: flex;
flex-direction: column;
justify-content: center;
align-items: center;
z-index: 50;
transition: opacity 0.8s ease;
}
.loader-text {
margin-top: 20px;
font-size: 14px;
color: var(--text-secondary);
font-weight: 500;
}
/* Onboarding Overlay */
.onboarding-overlay {
position: absolute;
top: 50%;
left: 50%;
transform: translate(-50%, -50%);
text-align: center;
opacity: 0;
transition: opacity 0.5s ease;
pointer-events: none;
}
.onboarding-overlay.visible {
opacity: 1;
}
.hand-outline {
width: 120px;
height: 180px;
border: 2px dashed rgba(255, 255, 255, 0.3);
border-radius: 40px;
margin: 0 auto 20px;
position: relative;
}
.hand-outline::after {
content: '';
position: absolute;
top: 20%; left: 20%; right: 20%; bottom: 20%;
border: 2px solid rgba(255, 255, 255, 0.5);
border-radius: 20px;
animation: pulse 2s infinite;
}
@keyframes pulse {
0% { transform: scale(0.95); opacity: 0.5; }
50% { transform: scale(1.05); opacity: 1; }
100% { transform: scale(0.95); opacity: 0.5; }
}
/* Drawing Mode Indicator */
.mode-indicator {
position: absolute;
top: 120px; /* Moved down below the buttons */
left: 50%;
transform: translateX(-50%);
background: rgba(0, 0, 0, 0.6);
backdrop-filter: blur(20px);
padding: 8px 16px;
border-radius: 99px;
font-size: 14px;
font-weight: 600;
opacity: 0;
transition: opacity 0.3s;
}
.mode-indicator.visible {
opacity: 1;
}
</style>
</head>
<body>
<div class="immersive-container">
<!-- Core Layers -->
<video id="video" autoplay muted playsinline></video>
<canvas id="canvas"></canvas>
<canvas id="drawingCanvas"></canvas>
<!-- Virtual Cursor -->
<div id="cursor" class="cursor"></div>
<!-- UI Layer -->
<div class="ui-layer">
<div class="hud-top-left">
<div class="hud-card" id="fpsCard">
<div class="hud-label">FPS</div>
<div class="hud-value" id="fpsValue">0</div>
</div>
<div class="hud-card" id="gestureCard">
<div class="hud-label">Gesture</div>
<div class="hud-value" id="gestureValue">None</div>
</div>
</div>
<div class="mode-indicator" id="modeIndicator">Mode: Cursor</div>
<!-- Music Panel (Hidden by default) -->
<div id="musicPanel">
<div class="music-header">
<span>🎵 Music Library</span>
</div>
<div class="playlist" id="playlist">
<!-- Tracks injected here -->
</div>
<div class="mini-player">
<div class="play-btn" id="mainPlayBtn">▶</div>
<div class="track-info">
<div class="track-title" id="currentTitle">Not Playing</div>
<div class="track-artist" id="currentArtist">-</div>
</div>
</div>
</div>
<!-- Interactive Dock -->
<div class="interactive-area">
<div class="action-btn" id="btnPaint" data-mode="paint">绘图</div>
<div class="action-btn" id="btnMusic" data-mode="music">音乐</div>
<div class="action-btn" id="btnClear" data-mode="clear">清除</div>
</div>
<!-- Onboarding -->
<div class="onboarding-overlay" id="onboarding">
<div class="hand-outline"></div>
<div style="color: rgba(255,255,255,0.8); font-size: 16px;">Place Hand in View</div>
</div>
<!-- Copyright Footer -->
<div class="copyright-footer">
<a href="https://yundev.cn" target="_blank">研云代码</a>&<a href="https://halosb.com" target="_blank">知栖小筑</a> 作者:<a href="mailto:i@halosb.com">芝麻</a>
</div>
</div>
<!-- Loader -->
<div class="loader-overlay" id="loader">
<div style="color:white; font-size: 24px;"> Vision Hand OS</div>
<div class="loader-text">Initializing Neural Engine...</div>
</div>
</div>
<!-- Libraries -->
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-core@4.14.0/dist/tf-core.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-backend-webgl@4.14.0/dist/tf-backend-webgl.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@mediapipe/hands@0.4.1646424915/hands.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@tensorflow-models/hand-pose-detection@2.0.0/dist/hand-pose-detection.min.js"></script>
<script>
/**
* 1€ Filter Implementation for Jitter Reduction
*/
class OneEuroFilter {
constructor(minCutoff = 1.0, beta = 0.0, dcutoff = 1.0) {
this.minCutoff = minCutoff;
this.beta = beta;
this.dcutoff = dcutoff;
this.x = null;
this.dx = null;
this.lastTime = null;
}
alpha(cutoff) {
const te = 1.0 / 60.0;
const tau = 1.0 / (2 * Math.PI * cutoff);
return 1.0 / (1.0 + tau / te);
}
filter(value, timestamp) {
const dx = (this.x === null) ? 0 : (value - this.x);
const edx = (this.dx === null) ? dx : (this.alpha(this.dcutoff) * dx + (1 - this.alpha(this.dcutoff)) * this.dx);
this.dx = edx;
const cutoff = this.minCutoff + this.beta * Math.abs(edx);
const x = (this.x === null) ? value : (this.alpha(cutoff) * value + (1 - this.alpha(cutoff)) * this.x);
this.x = x;
return x;
}
}
// --- Core Variables ---
const video = document.getElementById('video');
const canvas = document.getElementById('canvas');
const drawingCanvas = document.getElementById('drawingCanvas');
const ctx = canvas.getContext('2d', { alpha: true });
const drawCtx = drawingCanvas.getContext('2d', { alpha: true });
const cursor = document.getElementById('cursor');
const gestureValue = document.getElementById('gestureValue');
const fpsValue = document.getElementById('fpsValue');
const musicPanel = document.getElementById('musicPanel');
const playlistEl = document.getElementById('playlist');
const modeIndicator = document.getElementById('modeIndicator');
// App State
const STATE = {
mode: 'cursor', // 'cursor', 'paint', 'music'
isPinching: false,
pinchStart: { x: 0, y: 0 },
scrollOffset: 0,
color: '#0A84FF',
playlist: [],
currentTrackIndex: -1,
lastPinchTime: 0,
drawPath: [] // For smooth curves
};
// Music API Config
const API_BASE = 'https://api.qijieya.cn/meting/';
const SERVER = 'netease';
const PLAYLIST_ID = '13789363449';
const audio = new Audio();
// Filters
let filters = [];
for (let i = 0; i < 21; i++) {
filters.push({ x: new OneEuroFilter(0.5, 0.05), y: new OneEuroFilter(0.5, 0.05), z: new OneEuroFilter(0.5, 0.05) });
}
// --- Initialization ---
async function init() {
try {
await tf.setBackend('webgl');
const stream = await navigator.mediaDevices.getUserMedia({
video: { facingMode: 'user', width: { ideal: 1920 }, height: { ideal: 1080 } }
});
video.srcObject = stream;
await new Promise(r => video.onloadedmetadata = r);
video.play();
resize();
const model = handPoseDetection.SupportedModels.MediaPipeHands;
const detector = await handPoseDetection.createDetector(model, {
runtime: 'mediapipe',
solutionPath: 'https://cdn.jsdelivr.net/npm/@mediapipe/hands@0.4.1646424915/',
modelType: 'full',
maxHands: 1
});
// Init Playlist
initPlaylist();
// Start Loop
document.getElementById('loader').style.opacity = 0;
setTimeout(() => document.getElementById('loader').remove(), 800);
loop(detector);
} catch (e) {
console.error(e);
alert("Init Failed: " + e.message);
}
}
function resize() {
canvas.width = window.innerWidth;
canvas.height = window.innerHeight;
drawingCanvas.width = window.innerWidth;
drawingCanvas.height = window.innerHeight;
// Re-setup drawing context after resize
drawCtx.lineCap = 'round';
drawCtx.lineJoin = 'round';
drawCtx.lineWidth = 5;
drawCtx.strokeStyle = STATE.color;
}
window.addEventListener('resize', resize);
// --- Music Player Logic ---
async function fetchJson(url) {
const res = await fetch(url);
return res.json();
}
async function initPlaylist() {
try {
const data = await fetchJson(`${API_BASE}?server=${SERVER}&type=playlist&id=${PLAYLIST_ID}`);
STATE.playlist = Array.isArray(data) ? data : (data.data || []);
renderPlaylist();
} catch(e) { console.error('Playlist Error', e); }
}
function renderPlaylist() {
playlistEl.innerHTML = '';
STATE.playlist.forEach((track, i) => {
const div = document.createElement('div');
div.className = `track-item ${i === STATE.currentTrackIndex ? 'active' : ''}`;
div.innerHTML = `
<div class="track-info">
<div class="track-title">${track.title || track.name}</div>
<div class="track-artist">${track.author || track.artist}</div>
</div>
`;
div.id = `track-${i}`;
playlistEl.appendChild(div);
});
}
async function playTrack(index) {
if (index < 0 || index >= STATE.playlist.length) return;
STATE.currentTrackIndex = index;
renderPlaylist();
const track = STATE.playlist[index];
document.getElementById('currentTitle').innerText = track.title || track.name;
document.getElementById('currentArtist').innerText = track.author || track.artist;
// Fetch URL if needed
let url = track.url;
if (!url && (track.id || track.songid)) {
const res = await fetchJson(`${API_BASE}?server=${SERVER}&type=url&id=${track.id || track.songid}`);
url = res.url;
}
if (url) {
audio.src = url;
audio.play();
document.getElementById('mainPlayBtn').innerText = '⏸';
}
}
// --- Main Loop ---
let lastTime = 0;
let frameCount = 0;
let lastFpsTime = 0;
async function loop(detector) {
const timestamp = performance.now();
// FPS
frameCount++;
if (timestamp - lastFpsTime >= 1000) {
fpsValue.innerText = frameCount;
frameCount = 0;
lastFpsTime = timestamp;
}
if (video.readyState >= 2) {
try {
const hands = await detector.estimateHands(video, { flipHorizontal: false });
ctx.clearRect(0, 0, canvas.width, canvas.height);
if (hands.length > 0) {
const hand = hands[0];
// 1. Smooth
const keypoints = hand.keypoints.map((kp, i) => ({
x: filters[i].x.filter(kp.x, timestamp),
y: filters[i].y.filter(kp.y, timestamp),
z: kp.z ? filters[i].z.filter(kp.z, timestamp) : 0
}));
// 2. Render Hand
drawHand(keypoints);
// 3. Logic
handleInteraction(keypoints, timestamp);
document.getElementById('onboarding').classList.remove('visible');
} else {
document.getElementById('onboarding').classList.add('visible');
cursor.classList.remove('visible');
STATE.isPinching = false;
STATE.drawPath = []; // Clear path on exit
}
} catch(e) { console.error(e); }
}
requestAnimationFrame(() => loop(detector));
}
// --- Interaction Logic ---
function handleInteraction(keypoints, timestamp) {
const indexTip = keypoints[8];
const thumbTip = keypoints[4];
// Map coordinates (Mirroring X)
const x = window.innerWidth - (indexTip.x / video.videoWidth) * window.innerWidth;
const y = (indexTip.y / video.videoHeight) * window.innerHeight;
// Update Cursor
cursor.style.left = x + 'px';
cursor.style.top = y + 'px';
cursor.classList.add('visible');
// Detect Pinch - Optimized Threshold
const dist = Math.hypot(thumbTip.x - indexTip.x, thumbTip.y - indexTip.y);
const wrist = keypoints[0];
const middleMCP = keypoints[9];
const handSize = Math.hypot(wrist.x - middleMCP.x, wrist.y - middleMCP.y);
const pinchDist = dist / handSize;
// Slightly hysteresis for stability
const pinchThresholdStart = 0.12;
const pinchThresholdEnd = 0.15;
let isPinching = STATE.isPinching;
if (!STATE.isPinching && pinchDist < pinchThresholdStart) {
isPinching = true;
} else if (STATE.isPinching && pinchDist > pinchThresholdEnd) {
isPinching = false;
}
// State Machine
if (isPinching && !STATE.isPinching) {
// Pinch Start
STATE.isPinching = true;
STATE.pinchStart = { x, y };
cursor.classList.add('clicking');
gestureValue.innerText = 'Pinch';
// Trigger Actions
const buttonClicked = checkClicks(x, y);
if (STATE.mode === 'music') {
checkMusicClicks(x, y);
}
// Drawing Start
if (STATE.mode === 'paint' && !buttonClicked) {
STATE.drawPath = [{x, y}];
}
} else if (isPinching && STATE.isPinching) {
// Dragging / Holding
if (STATE.mode === 'paint') {
paintSmooth(x, y);
} else if (STATE.mode === 'music') {
scrollPlaylist(y - STATE.pinchStart.y);
}
// Update start for delta calc
STATE.pinchStart = { x, y };
} else if (!isPinching && STATE.isPinching) {
// Release
STATE.isPinching = false;
cursor.classList.remove('clicking');
gestureValue.innerText = 'None';
// Check for Tap (vs Drag)
const dragDist = Math.hypot(x - STATE.gestureStart.x, y - STATE.gestureStart.y);
if (dragDist < 30 && STATE.mode === 'music') {
// It was a tap, trigger click at original position
checkMusicClicks(STATE.gestureStart.x, STATE.gestureStart.y);
}
if (STATE.mode === 'paint') {
STATE.drawPath = []; // End path
}
}
// Hover Effects
checkHover(x, y);
}
function checkClicks(x, y) {
// Check dock buttons
const buttons = document.querySelectorAll('.action-btn');
buttons.forEach(btn => {
const rect = btn.getBoundingClientRect();
const dist = Math.hypot(rect.left + rect.width/2 - x, rect.top + rect.height/2 - y);
if (dist < 40) {
const mode = btn.dataset.mode;
if (mode === 'clear') {
drawCtx.clearRect(0, 0, drawingCanvas.width, drawingCanvas.height);
} else {
switchMode(mode);
// Visual toggle
document.querySelectorAll('.action-btn').forEach(b => b.classList.remove('active'));
btn.classList.add('active');
}
triggerHaptic();
}
});
}
function checkMusicClicks(x, y) {
// Check if clicked inside playlist items
if (!musicPanel.classList.contains('visible')) return;
// Check main controls
const buttons = document.querySelectorAll('.action-btn'); // Still check dock to switch back
buttons.forEach(btn => {
const rect = btn.getBoundingClientRect();
const dist = Math.hypot(rect.left + rect.width/2 - x, rect.top + rect.height/2 - y);
if (dist < 40) {
const mode = btn.dataset.mode;
if (mode !== 'clear') {
switchMode(mode);
document.querySelectorAll('.action-btn').forEach(b => b.classList.remove('active'));
btn.classList.add('active');
}
}
});
// Check track items
const tracks = document.querySelectorAll('.track-item');
tracks.forEach((track, i) => {
const rect = track.getBoundingClientRect();
if (x >= rect.left && x <= rect.right && y >= rect.top && y <= rect.bottom) {
playTrack(i);
triggerHaptic();
}
});
}
function checkHover(x, y) {
const allClickables = [...document.querySelectorAll('.action-btn'), ...document.querySelectorAll('.track-item')];
allClickables.forEach(el => {
const rect = el.getBoundingClientRect();
// Simple rect hit test for list items, circle for buttons
let hit = false;
if (el.classList.contains('action-btn')) {
const dist = Math.hypot(rect.left + rect.width/2 - x, rect.top + rect.height/2 - y);
hit = dist < 40;
} else {
hit = (x >= rect.left && x <= rect.right && y >= rect.top && y <= rect.bottom);
}
if (hit) el.classList.add('hovered');
else el.classList.remove('hovered');
});
}
function switchMode(newMode) {
STATE.mode = newMode;
modeIndicator.innerText = 'Mode: ' + newMode.charAt(0).toUpperCase() + newMode.slice(1);
modeIndicator.classList.add('visible');
setTimeout(() => modeIndicator.classList.remove('visible'), 2000);
if (newMode === 'paint') {
drawingCanvas.classList.add('active');
musicPanel.classList.remove('visible');
cursor.style.borderColor = STATE.color;
} else if (newMode === 'music') {
drawingCanvas.classList.remove('active');
musicPanel.classList.add('visible');
cursor.style.borderColor = '#fff';
} else {
drawingCanvas.classList.remove('active');
musicPanel.classList.remove('visible');
cursor.style.borderColor = '#fff';
}
}
// Optimized Smooth Painting using Quadratic Curves
function paintSmooth(x, y) {
STATE.drawPath.push({x, y});
// Need at least 3 points to draw a smooth curve
if (STATE.drawPath.length < 3) return;
const path = STATE.drawPath;
const len = path.length;
// Get the last 3 points
const p0 = path[len - 3];
const p1 = path[len - 2];
const p2 = path[len - 1];
drawCtx.lineWidth = 5;
drawCtx.lineCap = 'round';
drawCtx.lineJoin = 'round';
drawCtx.strokeStyle = STATE.color;
drawCtx.beginPath();
drawCtx.moveTo(p0.x, p0.y);
// Use quadratic curve for smoothing between points
// Control point is p1, end point is mid of p1-p2
// Actually, better smoothing: mid points
// Simple Quadratic Bezier:
// Start at mid(p0, p1), control p1, end mid(p1, p2)
// This ensures continuity
// But since we are drawing incrementally, we can just draw from p0 to p1?
// No, straight lines look jagged.
// Incremental Smooth Drawing:
// We draw the curve segment from the previous mid-point to the current mid-point.
const mid1 = { x: (p0.x + p1.x) / 2, y: (p0.y + p1.y) / 2 };
const mid2 = { x: (p1.x + p2.x) / 2, y: (p1.y + p2.y) / 2 };
drawCtx.beginPath();
drawCtx.moveTo(mid1.x, mid1.y);
drawCtx.quadraticCurveTo(p1.x, p1.y, mid2.x, mid2.y);
drawCtx.stroke();
}
function scrollPlaylist(deltaY) {
const playlist = document.getElementById('playlist');
playlist.scrollTop -= deltaY * 2; // Speed multiplier
}
function drawHand(keypoints) {
const connections = [[0,1],[1,2],[2,3],[3,4],[0,5],[5,6],[6,7],[7,8],[5,9],[9,10],[10,11],[11,12],[9,13],[13,14],[14,15],[15,16],[13,17],[0,17],[17,18],[18,19],[19,20]];
ctx.lineCap = 'round';
ctx.lineJoin = 'round';
connections.forEach(([i, j]) => {
const p1 = keypoints[i];
const p2 = keypoints[j];
const x1 = window.innerWidth - (p1.x / video.videoWidth) * window.innerWidth;
const y1 = (p1.y / video.videoHeight) * window.innerHeight;
const x2 = window.innerWidth - (p2.x / video.videoWidth) * window.innerWidth;
const y2 = (p2.y / video.videoHeight) * window.innerHeight;
const depth = (p1.z || 0);
const opacity = Math.max(0.2, Math.min(0.8, 1 + depth * 5));
ctx.beginPath();
ctx.moveTo(x1, y1);
ctx.lineTo(x2, y2);
ctx.lineWidth = 3;
ctx.strokeStyle = `rgba(255, 255, 255, ${opacity})`;
ctx.stroke();
});
}
function triggerHaptic() {
if (navigator.vibrate) navigator.vibrate(10);
}
// Start
window.addEventListener('load', init);
</script>
</body>
</html> }
// Start
window.addEventListener('load', init);
</script>
</body>
</html>| 技术栈: TensorFlow.js, MediaPipe, HTML5 Canvas
本文是原创文章，采用 CC BY-NC-ND 4.0 协议，完整转载请注明来自 研云代码
评论
匿名评论
隐私政策
你无需删除空行，直接评论以获取最佳展示效果
