Hey, welcome back!

Pichle hafte maine promise kiya tha ki hum AI ko bilkul basics se samjhenge. Aaj ka topic simple hai, par trust me, iske baad aapko har chatbot alag dikhega. Main wahi explain karunga jo main chahta tha ki mujhe koi pehle din samjha deta.

In this newsletter

Ek chhota sa game jo AI roz khelta hai
"Surprise" ko number mein kaise naapte hain
Ek trap jo sabko confuse karta hai
Agle hafte kya aa raha hai

1. Ek chhota sa game

Main ek sentence bolta hoon: "Aaj raat ko main ghar jaakar khaana ____". Aap bina soche bol doge: "khaunga".

Aapne abhi wahi kiya jo ek language model karta hai. Wo bhi agla word guess karta hai, bas bahut zyada text padh kar. Did you get the pain? Jisko hum "intelligence" kehte hain, uska bada hissa yeh guessing game hi hai.

2. Surprise ko number kaise banate hain

Claude Shannon (Bell Labs ke scientist) ne 1948 mein "A Mathematical Theory of Communication" paper likha. Usme unhone ek idea diya jise entropy kehte hain, matlab "kitna surprise hai".

Ek fair coin socho, jisme head aur tail dono 50-50 hain. Uska result pehle se guess karna mushkil hai, isliye entropy 1 bit hoti hai.

Ab ek unfair coin socho, jisme 90% head aata hai. Yahan aap aaram se "head" guess kar sakte ho, isliye surprise kam hai.

Fair coin (50/50):    1 bit
Unfair coin (90/10):  about 0.47 bit

Rule simple hai: jitna predict karna aasaan, utni entropy kam.

3. The trap that catches everyone

⚠️ Entropy ka matlab "information ki quality" nahi hota. Entropy sirf yeh batati hai ki next result kitna unpredictable hai.

Ek random gibberish string ki entropy bahut high hoti hai, par usme koi matlab nahi hota. Matlab aur surprise alag cheezein hain.

Fix yeh hai: jab bhi "entropy" sunte ho, bas yeh poocho, "kitna predict kar sakte hain?" Isse zyada kuch mat maano.

4. Let's be honest about what just happened

Humne bina heavy maths ke ek bade idea ko chhua. AI guess karta hai, aur Shannon ne is guess ko naapne ka tareeka diya. Humne abhi sirf coin dekha hai. Asli language model mein kaafi aur layers hain.

Next week: hum dekhenge ki ek simple "n-gram" model kaise agla word guess karta hai, bas word counting se. Tighten your seatbelt.

Aur padhna ho toh

From Shannon to Transformers: poori series, jahan 1948 aur 1951 dono Shannon papers step by step samjhaye hain.
unicodesnwrites blog, saare articles ek jagah.

Milte hain agle hafte,
Unique

Want the next email?