#loss

Calculating Loss In Social Media

How not to mix things up in a world full of casual brushes with the truth – uncaring as to what is real and what is not. In principle it has always been the same for at least some of the population. Now, however, we have reached critical mass. Each uncaring and unthinking person has access to weapons of war. A tweet ricochets off another tweet and spins off into another social media platform. Like a pinball machine full of pinballs, the machine gets hot and melts down. Anger, frustration, good old-fashioned gratuitous annoyance – they have nowhere they want to go except deeper into the furnace

Brushes With The Truth · EN

Why Is the Loss Function of Large Models Cross-Entropy

Prologue When first entering the world of large models, due to gaps in foundational math knowledge such as linear algebra, probability theory, and information theory, it’s easy to get lost among numerous terms: logprob (log probability), likelihood, NLL (Negative Log Likelihood), cross entropy, perplexity. They frequently appear in various corners of papers and documentation, yet they all feel like acquaintances you only know by name — you see them often, but don’t truly understand them. Then one day, after slowly catching up on some basic math knowledge and immersing myself in the company con

木鸟杂记 · EN

大模型的损失函数为什么是交叉熵

引子 刚入门大模型的时候，由于线性代数、概率论和信息论等数学知识的短板，很容易迷失在诸多术语中：logprob（对数概率）、likelihood（似然）、NLL（Negative Log Likelihood，负对数似然）、cross entropy（交叉熵）、perplexity（困惑度）。它们常常出现在论文和文档的各种角落里，但都像点赞之交的朋友，频见其名，不解其意。 后来某天，在慢慢的补过一些最基础的数学知识后，在公司相关的上下文浸淫足够久后，终于在某次和 ChatGPT 的聊天中发现： 上面一组概念本质上是同一件事的不同面向的侧写 。从概率论的门摸进去叫 NLL，从信息论的门踏进去叫交叉熵，从 PyTorch 的门看进去叫 F.cross_entropy ——殊途同归，本质上都是在试图刻画"模型当前输出离预期还有多远"。 “横看成岭侧成峰”，在大模型这种高维上下文的领域中，这种盲人摸象的感觉所在多有。不过我们这种三维生物，也只能靠长久的浸淫，才能靠着不同领域知识的交叉验证，才会突然有一天顿悟——嗷，原来这是同一座山。 本文想做的，就是想聊聊大模型领域中最基础概念——交叉熵这个损失函数的“一花各表”。

木鸟杂记 · ZH

【译文】/goal + 损失函数：如何用一条指令在 30 小时内蒸馏一个产品

原文 ： /goal + Loss Functions: How to Distill a Product in 30 Hours with One Prompt [Full Playbook] 作者 ：Elvis ( @elvissun ) 翻译 ：AI 辅助翻译 99% 的人用 /goal 和 agent loop 的方式是错的。 外面吹的是「长循环自主 agent：设定目标，走开，回来就拿到能跑的代码」。但顶尖的 agent 工程师半年前就在做同样的事了——用 harness engineering + spec-driven development： 给 agent 搭一个能观察问题的测试框架 写一份包含所有 test case 的 tight spec 让 Codex 或 Claude Code 无人值守循环，直到所有测试通过 我经常睡前 kick off 这种任务——一次跑 2-5 小时。四月份有一次，它啃了一整晚我们 Vercel monorepo 里的 Turbo build-cache bug，天亮时全绿了。根本不需要 /goal 。 所以 /goal 到底用来干什么？ 这是我离开后一条 prompt 干的事： 跑了约 30 小时，生成 6300 行代码，爬了 92000 页，花了 $40 API 把另一个产品的核心逻辑反向工程并完整重构 我们的版本输出质量比参考

电波障害 · ZH

Don't Be Afraid

A tormented photographer makes a comeback with freshly taken photos at the very location where his son once disappeared. During the exhibition, visitors suddenly realize that there's someone lurking in the background of several photos.

Short of the Week · EN