دانلود کتاب ارزیابی مدل‌های زبانی بزرگ (LLM) در مقیاس وسیع: آزمون، تشخیص رگرسیون و سنجش کیفیت برای سیستم‌های هوش مصنوعی مولد

The dashboards were green. All of them. Latency was inside budget, error rates were flat, the deployment pipeline had gone through without a single failed check, and the on-call engineer had already gone to bed. It was a little after two in the morning, and by every measure the engineering organization had agreed to care about, the release was a success. Four days later, a product manager forwarded a screenshot. A customer had asked the assistant to compare three products. The assistant had listed them cleanly. The customer had then typed a short follow-up, six words, asking about the second one. The assistant answered about the first. The customer corrected it. The assistant apologized and answered about the third. The screenshot ended there, because that is where the customer had ended the conversation and, most likely, ended their patience with the product. Nothing in our monitoring had fired. No exception was thrown. No timeout occurred. The service returned a well-formed response in under a second, with a correct HTTP status code and a payload that passed schema validation. Every test we had written was still passing. The system was, in the precise engineering sense of the word, working. That screenshot has stayed with me longer than any outage I have been paged for. Outages are honest. They tell you they happened. This kind of failure does not announce itself. It sits quietly inside a healthy-looking system, degrading the product one conversation at a time, and it will keep doing so for as long as your instruments have nothing to say about it.