大模型、多模态、推理模型和 AI 产品趋势
This collection features articles and community discussions in the AI 前沿 space, focusing on real-world engineering scenarios, tool selection, practical paths, and risk boundaries.
The page is continuously updated and serves as a learning roadmap, technology selection reference, and technical review entry point.
Making a model emit JSON is a common AI engineering need and failure point: an extra field, missing comma, or stray explanation breaks parsing. This article presents four tiers: prompt constraints, JSON mode, Function Calling, and constrained decoding (grammar), plus parsing and recovery safeguards that turn "usually works" into production reliability.
KV Cache is not leftover GPU memory allocated first come, first served. This guide designs measurable admission and eviction controls around per-request memory estimates, tenant budgets, block allocation, preemption costs, and prefix isolation.
Fitting context into the window does not make the message sequence valid. This article binds roles, tool calls/results, serialized requests, and streaming blocks into one dispatch manifest so partial tool arguments or stale streams cannot leak into the next turn.
Ad hoc truncation after a context overflow turns a cost problem into an incorrect answer. This article makes context assembly a server-side admission protocol, controlling each call through bucketed reservations, a deterministic degradation ladder, and settlement metrics.
Correlation between a Judge and human ratings does not mean it can compare new models reliably. This article turns blinded anchors, position swaps, style counterfactuals, calibration mappings, and Judge releases into a reproducible admission contract.
Evidence that works at the beginning may be missed in the middle. Build position-controlled metamorphic cases and block model or context-assembly regressions using worst-position performance rather than averages.
APIs are convenient, but compliance, cost, offline operation, and customization can require self-hosting. This guide compares Ollama for local experiments and personal development with vLLM for high-throughput production inference, then covers quantization, VRAM, hardware selection, and when to use an API instead.
Dynamically loading multiple LoRA adapters onto one base model saves GPU capacity but introduces artifact substitution, cache contamination, and cross-tenant mix-up risks. This article designs trusted registration, resource limits, batching isolation, and per-adapter rollback.
Major foundation models now support images natively, turning image understanding from a toy into a product capability. This article explains Vision strengths, typical use cases (document parsing, support, moderation, accessibility), and engineering essentials: image token costs, preprocessing, prompt design, limitations, and pitfalls, so you can decide when
A higher hit rate does not prove cache correctness. This article defines scopes, content identity, authorization epochs, hit revalidation, and cross-tenant negative tests for managed and self-hosted prefix caches.
A successful upload does not mean media can be sent directly to a model. This article defines contracts for image, audio, and video probing, decoding, limits, orientation and color correction, normalization manifests, and version rollback.
Offline scores can rise while online satisfaction falls because of population shifts, delayed labels, fallbacks, or multiple component upgrades. This article freezes exposures, complete runtime bundles, label watermarks, and attribution windows into evidence that can be corrected but not rewritten.