[{"data":1,"prerenderedAt":10466},["ShallowReactive",2],{"\u002Fblog\u002FData-Preprocessing-&-Regression-Analysis":3,"post-count":10058,"series-global-data":10059,"authors-all":10169,"series-sidebar-none":10364,"sidebar-authors":10365},{"id":4,"title":5,"author":6,"body":7,"date":10047,"description":5496,"draft":10048,"edited_at":10047,"extension":10049,"featured_image":10050,"meta":10051,"navigation":10052,"path":10053,"pinned":10048,"seo":10054,"sitemap":10055,"stem":10056,"tags":10050,"__hash__":10057},"blog\u002Fblog\u002FData-Preprocessing-&-Regression-Analysis.md","Data Preprocessing & Regression Analysis","chinono",{"type":8,"value":9,"toc":10025},"minimark",[10,15,24,27,32,38,56,62,68,98,104,126,132,492,506,512,516,519,541,545,548,552,558,569,588,592,595,897,944,948,1014,1170,1225,1229,1236,1446,1543,1567,1571,1574,1715,1872,1926,1957,1960,2164,2175,5312,5316,5326,5332,5362,5368,5372,5375,5429,5435,5442,5446,5453,5456,5671,5702,6919,6923,6925,6932,6943,7088,7171,7970,7974,7981,8262,8265,8590,8601,8790,8793,9937,9940,10021],[11,12,14],"h2",{"id":13},"data-preprocessing","Data Preprocessing",[16,17,18,19,23],"p",{},"Before we can train any Machine Learning model, we need to deal with the messy reality of raw data. ",[20,21,22],"strong",{},"Data preprocessing"," is the process of transforming raw data into a clean, usable format before feeding it into a model. Think of it like washing and chopping vegetables before cooking. You can't make a good dish with dirty, uncut ingredients.",[16,25,26],{},"Raw data typically suffers from problems like outliers, missing values, meaningless combinations, and inconsistent scales. Preprocessing turns that chaos into something a model can actually learn from.",[28,29,31],"h3",{"id":30},"the-7-steps-of-data-preprocessing","The 7 Steps of Data Preprocessing",[16,33,34,37],{},[20,35,36],{},"Step 1 — Acquire the dataset."," Gather data from various sources and combine them into a proper format. The structure of your dataset will depend on your domain (business data looks very different from medical data).",[16,39,40,43,44,47,48,51,52,55],{},[20,41,42],{},"Step 2 — Import crucial libraries."," In Python, three libraries form the backbone of preprocessing: ",[20,45,46],{},"NumPy"," (scientific computing), ",[20,49,50],{},"Pandas"," (data manipulation and analysis), and ",[20,53,54],{},"Matplotlib"," (2D plotting and visualization).",[16,57,58,61],{},[20,59,60],{},"Step 3 — Import the dataset."," Load your data into your working environment. Make sure your working directory is set correctly so your code can find the files.",[16,63,64,67],{},[20,65,66],{},"Step 4 — Handle missing values."," This is critical. If you ignore missing data, your model will draw faulty conclusions. Common strategies include filling missing values with the column mean\u002Fmedian, or dropping incomplete rows entirely.",[69,70,74,78],"details",{"className":71},[72,73],"info-box","info-box-info",[75,76,77],"summary",{},"What's imputation (filling in) and dropping (removing row)?",[79,80,83,95],"div",{"className":81},[82],"info-box-content",[16,84,85,86,90,91,94],{},"Imagine a spreadsheet of patient records where some rows have no entry for \"blood pressure.\" If you feed that gap straight into a model, it either crashes or silently learns something wrong. You have two main options: ",[87,88,89],"em",{},"imputation"," (filling in a reasonable substitute) or ",[87,92,93],{},"deletion"," (removing the incomplete row).",[16,96,97],{},"For imputation, the most common approach is replacing the blank with the column's mean or median. Mean works well for normally distributed data, while median is safer when you have outliers — for instance, if most blood pressures are around 120 but one reading is 300, the mean gets pulled up, while the median stays stable. Deletion is simpler but risky: if 30% of your rows are missing that field, you're throwing away a lot of data.",[16,99,100,103],{},[20,101,102],{},"Step 5 — Encode categorical data."," ML models run on math — they need numbers, not labels like \"Red\" or \"Male.\" Encoding methods include dummy variables, label encoding (assigning integers to categories), and one-hot encoding (creating binary columns for each category).",[69,105,107,110],{"className":106},[72,73],[75,108,109],{},"Label encoding vs one-hot encoding",[79,111,113,120],{"className":112},[82],[16,114,115,116,119],{},"Models do math, so a column like \"Color\" with values Red, Blue, Green is meaningless to them as text. ",[87,117,118],{},"Label encoding"," assigns each category an integer (Red=0, Blue=1, Green=2), which is quick but introduces a problem: the model may interpret Blue as \"between\" Red and Green, or assume Green > Red, when there's no such ordering.",[16,121,122,125],{},[87,123,124],{},"One-hot encoding"," avoids this by creating a separate binary column for each category, so you'd get three columns (is_Red, is_Blue, is_Green), each containing 0 or 1. The trade-off is that one-hot encoding adds more columns, which can be expensive with high-cardinality features (imagine a \"City\" column with 500 unique values).",[16,127,128,131],{},[20,129,130],{},"Step 6 — Feature scaling."," When your features have wildly different ranges (e.g., age 0–100 vs. salary 0–1,000,000), some algorithms get confused. Feature scaling standardizes variables to a common range. Two popular methods are Min-Max scaling and Standardisation (z-score).",[69,133,135,138],{"className":134},[72,73],[75,136,137],{},"Feature scaling",[79,139,141,144,489],{"className":140},[82],[16,142,143],{},"Say you have two features: age (ranging 18–65) and income (ranging 20,000–200,000). A distance-based algorithm like k-nearest neighbours calculates how \"close\" two data points are. Without scaling, income dominates that distance calculation simply because its numbers are bigger — age differences become basically invisible. Scaling fixes this by putting both features on a comparable range.",[16,145,146,149,150,325,326,329,330,488],{},[87,147,148],{},"Min-Max scaling"," squeezes everything into 0–1 using the formula ",[151,152,156],"mjx-container",{"className":153,"jax":155},[154],"MathJax","SVG",[157,158,167,207],"svg",{"style":159,"xmlns":160,"width":161,"height":162,"role":163,"focusable":164,"viewBox":165,"xmlnsXLink":166},"vertical-align: -0.566ex;","http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg","24.203ex","2.262ex","img","false","0 -750 10697.9 1000","http:\u002F\u002Fwww.w3.org\u002F1999\u002Fxlink",[168,169,170,175,179,183,187,191,195,199,203],"defs",{},[171,172],"path",{"id":173,"d":174},"MJX-1-TEX-N-28","M94 250Q94 319 104 381T127 488T164 576T202 643T244 695T277 729T302 750H315H319Q333 750 333 741Q333 738 316 720T275 667T226 581T184 443T167 250T184 58T225 -81T274 -167T316 -220T333 -241Q333 -250 318 -250H315H302L274 -226Q180 -141 137 -14T94 250Z",[171,176],{"id":177,"d":178},"MJX-1-TEX-I-1D465","M52 289Q59 331 106 386T222 442Q257 442 286 424T329 379Q371 442 430 442Q467 442 494 420T522 361Q522 332 508 314T481 292T458 288Q439 288 427 299T415 328Q415 374 465 391Q454 404 425 404Q412 404 406 402Q368 386 350 336Q290 115 290 78Q290 50 306 38T341 26Q378 26 414 59T463 140Q466 150 469 151T485 153H489Q504 153 504 145Q504 144 502 134Q486 77 440 33T333 -11Q263 -11 227 52Q186 -10 133 -10H127Q78 -10 57 16T35 71Q35 103 54 123T99 143Q142 143 142 101Q142 81 130 66T107 46T94 41L91 40Q91 39 97 36T113 29T132 26Q168 26 194 71Q203 87 217 139T245 247T261 313Q266 340 266 352Q266 380 251 392T217 404Q177 404 142 372T93 290Q91 281 88 280T72 278H58Q52 284 52 289Z",[171,180],{"id":181,"d":182},"MJX-1-TEX-N-2212","M84 237T84 250T98 270H679Q694 262 694 250T679 230H98Q84 237 84 250Z",[171,184],{"id":185,"d":186},"MJX-1-TEX-I-1D45A","M21 287Q22 293 24 303T36 341T56 388T88 425T132 442T175 435T205 417T221 395T229 376L231 369Q231 367 232 367L243 378Q303 442 384 442Q401 442 415 440T441 433T460 423T475 411T485 398T493 385T497 373T500 364T502 357L510 367Q573 442 659 442Q713 442 746 415T780 336Q780 285 742 178T704 50Q705 36 709 31T724 26Q752 26 776 56T815 138Q818 149 821 151T837 153Q857 153 857 145Q857 144 853 130Q845 101 831 73T785 17T716 -10Q669 -10 648 17T627 73Q627 92 663 193T700 345Q700 404 656 404H651Q565 404 506 303L499 291L466 157Q433 26 428 16Q415 -11 385 -11Q372 -11 364 -4T353 8T350 18Q350 29 384 161L420 307Q423 322 423 345Q423 404 379 404H374Q288 404 229 303L222 291L189 157Q156 26 151 16Q138 -11 108 -11Q95 -11 87 -5T76 7T74 17Q74 30 112 181Q151 335 151 342Q154 357 154 369Q154 405 129 405Q107 405 92 377T69 316T57 280Q55 278 41 278H27Q21 284 21 287Z",[171,188],{"id":189,"d":190},"MJX-1-TEX-I-1D456","M184 600Q184 624 203 642T247 661Q265 661 277 649T290 619Q290 596 270 577T226 557Q211 557 198 567T184 600ZM21 287Q21 295 30 318T54 369T98 420T158 442Q197 442 223 419T250 357Q250 340 236 301T196 196T154 83Q149 61 149 51Q149 26 166 26Q175 26 185 29T208 43T235 78T260 137Q263 149 265 151T282 153Q302 153 302 143Q302 135 293 112T268 61T223 11T161 -11Q129 -11 102 10T74 74Q74 91 79 106T122 220Q160 321 166 341T173 380Q173 404 156 404H154Q124 404 99 371T61 287Q60 286 59 284T58 281T56 279T53 278T49 278T41 278H27Q21 284 21 287Z",[171,192],{"id":193,"d":194},"MJX-1-TEX-I-1D45B","M21 287Q22 293 24 303T36 341T56 388T89 425T135 442Q171 442 195 424T225 390T231 369Q231 367 232 367L243 378Q304 442 382 442Q436 442 469 415T503 336T465 179T427 52Q427 26 444 26Q450 26 453 27Q482 32 505 65T540 145Q542 153 560 153Q580 153 580 145Q580 144 576 130Q568 101 554 73T508 17T439 -10Q392 -10 371 17T350 73Q350 92 386 193T423 345Q423 404 379 404H374Q288 404 229 303L222 291L189 157Q156 26 151 16Q138 -11 108 -11Q95 -11 87 -5T76 7T74 17Q74 30 112 180T152 343Q153 348 153 366Q153 405 129 405Q91 405 66 305Q60 285 60 284Q58 278 41 278H27Q21 284 21 287Z",[171,196],{"id":197,"d":198},"MJX-1-TEX-N-29","M60 749L64 750Q69 750 74 750H86L114 726Q208 641 251 514T294 250Q294 182 284 119T261 12T224 -76T186 -143T145 -194T113 -227T90 -246Q87 -249 86 -250H74Q66 -250 63 -250T58 -247T55 -238Q56 -237 66 -225Q221 -64 221 250T66 725Q56 737 55 738Q55 746 60 749Z",[171,200],{"id":201,"d":202},"MJX-1-TEX-N-2F","M423 750Q432 750 438 744T444 730Q444 725 271 248T92 -240Q85 -250 75 -250Q68 -250 62 -245T56 -231Q56 -221 230 257T407 740Q411 750 423 750Z",[171,204],{"id":205,"d":206},"MJX-1-TEX-I-1D44E","M33 157Q33 258 109 349T280 441Q331 441 370 392Q386 422 416 422Q429 422 439 414T449 394Q449 381 412 234T374 68Q374 43 381 35T402 26Q411 27 422 35Q443 55 463 131Q469 151 473 152Q475 153 483 153H487Q506 153 506 144Q506 138 501 117T481 63T449 13Q436 0 417 -8Q409 -10 393 -10Q359 -10 336 5T306 36L300 51Q299 52 296 50Q294 48 292 46Q233 -10 172 -10Q117 -10 75 30T33 157ZM351 328Q351 334 346 350T323 385T277 405Q242 405 210 374T160 293Q131 214 119 129Q119 126 119 118T118 106Q118 61 136 44T179 26Q217 26 254 59T298 110Q300 114 325 217T351 328Z",[208,209,213],"g",{"stroke":210,"fill":210,"stroke-width":211,"transform":212},"currentColor","0","scale(1,-1)",[208,214,216,224,232,239,246,253,260,267,278,283,288,295,300,305,310,315,320],{"dataMmlNode":215},"math",[208,217,219],{"dataMmlNode":218},"mo",[220,221],"use",{"dataC":222,"xLinkHref":223},"28","#MJX-1-TEX-N-28",[208,225,228],{"dataMmlNode":226,"transform":227},"mi","translate(389,0)",[220,229],{"dataC":230,"xLinkHref":231},"1D465","#MJX-1-TEX-I-1D465",[208,233,235],{"dataMmlNode":218,"transform":234},"translate(1183.2,0)",[220,236],{"dataC":237,"xLinkHref":238},"2212","#MJX-1-TEX-N-2212",[208,240,242],{"dataMmlNode":226,"transform":241},"translate(2183.4,0)",[220,243],{"dataC":244,"xLinkHref":245},"1D45A","#MJX-1-TEX-I-1D45A",[208,247,249],{"dataMmlNode":226,"transform":248},"translate(3061.4,0)",[220,250],{"dataC":251,"xLinkHref":252},"1D456","#MJX-1-TEX-I-1D456",[208,254,256],{"dataMmlNode":226,"transform":255},"translate(3406.4,0)",[220,257],{"dataC":258,"xLinkHref":259},"1D45B","#MJX-1-TEX-I-1D45B",[208,261,263],{"dataMmlNode":218,"transform":262},"translate(4006.4,0)",[220,264],{"dataC":265,"xLinkHref":266},"29","#MJX-1-TEX-N-29",[208,268,272],{"dataMmlNode":269,"dataMjxTexclass":270,"transform":271},"TeXAtom","ORD","translate(4395.4,0)",[208,273,274],{"dataMmlNode":218},[220,275],{"dataC":276,"xLinkHref":277},"2F","#MJX-1-TEX-N-2F",[208,279,281],{"dataMmlNode":218,"transform":280},"translate(4895.4,0)",[220,282],{"dataC":222,"xLinkHref":223},[208,284,286],{"dataMmlNode":226,"transform":285},"translate(5284.4,0)",[220,287],{"dataC":244,"xLinkHref":245},[208,289,291],{"dataMmlNode":226,"transform":290},"translate(6162.4,0)",[220,292],{"dataC":293,"xLinkHref":294},"1D44E","#MJX-1-TEX-I-1D44E",[208,296,298],{"dataMmlNode":226,"transform":297},"translate(6691.4,0)",[220,299],{"dataC":230,"xLinkHref":231},[208,301,303],{"dataMmlNode":218,"transform":302},"translate(7485.7,0)",[220,304],{"dataC":237,"xLinkHref":238},[208,306,308],{"dataMmlNode":226,"transform":307},"translate(8485.9,0)",[220,309],{"dataC":244,"xLinkHref":245},[208,311,313],{"dataMmlNode":226,"transform":312},"translate(9363.9,0)",[220,314],{"dataC":251,"xLinkHref":252},[208,316,318],{"dataMmlNode":226,"transform":317},"translate(9708.9,0)",[220,319],{"dataC":258,"xLinkHref":259},[208,321,323],{"dataMmlNode":218,"transform":322},"translate(10308.9,0)",[220,324],{"dataC":265,"xLinkHref":266},". So an age of 18 becomes 0, an age of 65 becomes 1, and everything else falls between. ",[87,327,328],{},"Standardisation (z-score)"," instead centres the data around 0 with a standard deviation of 1, using ",[151,331,333],{"className":332,"jax":155},[154],[157,334,337,387],{"style":159,"xmlns":160,"width":335,"height":162,"role":163,"focusable":164,"viewBox":336,"xmlnsXLink":166},"20.06ex","0 -750 8866.4 1000",[168,338,339,342,345,348,351,355,358,361,364,367,371,375,379,383],{},[171,340],{"id":341,"d":174},"MJX-2-TEX-N-28",[171,343],{"id":344,"d":178},"MJX-2-TEX-I-1D465",[171,346],{"id":347,"d":182},"MJX-2-TEX-N-2212",[171,349],{"id":350,"d":186},"MJX-2-TEX-I-1D45A",[171,352],{"id":353,"d":354},"MJX-2-TEX-I-1D452","M39 168Q39 225 58 272T107 350T174 402T244 433T307 442H310Q355 442 388 420T421 355Q421 265 310 237Q261 224 176 223Q139 223 138 221Q138 219 132 186T125 128Q125 81 146 54T209 26T302 45T394 111Q403 121 406 121Q410 121 419 112T429 98T420 82T390 55T344 24T281 -1T205 -11Q126 -11 83 42T39 168ZM373 353Q367 405 305 405Q272 405 244 391T199 357T170 316T154 280T149 261Q149 260 169 260Q282 260 327 284T373 353Z",[171,356],{"id":357,"d":206},"MJX-2-TEX-I-1D44E",[171,359],{"id":360,"d":194},"MJX-2-TEX-I-1D45B",[171,362],{"id":363,"d":198},"MJX-2-TEX-N-29",[171,365],{"id":366,"d":202},"MJX-2-TEX-N-2F",[171,368],{"id":369,"d":370},"MJX-2-TEX-I-1D460","M131 289Q131 321 147 354T203 415T300 442Q362 442 390 415T419 355Q419 323 402 308T364 292Q351 292 340 300T328 326Q328 342 337 354T354 372T367 378Q368 378 368 379Q368 382 361 388T336 399T297 405Q249 405 227 379T204 326Q204 301 223 291T278 274T330 259Q396 230 396 163Q396 135 385 107T352 51T289 7T195 -10Q118 -10 86 19T53 87Q53 126 74 143T118 160Q133 160 146 151T160 120Q160 94 142 76T111 58Q109 57 108 57T107 55Q108 52 115 47T146 34T201 27Q237 27 263 38T301 66T318 97T323 122Q323 150 302 164T254 181T195 196T148 231Q131 256 131 289Z",[171,372],{"id":373,"d":374},"MJX-2-TEX-I-1D461","M26 385Q19 392 19 395Q19 399 22 411T27 425Q29 430 36 430T87 431H140L159 511Q162 522 166 540T173 566T179 586T187 603T197 615T211 624T229 626Q247 625 254 615T261 596Q261 589 252 549T232 470L222 433Q222 431 272 431H323Q330 424 330 420Q330 398 317 385H210L174 240Q135 80 135 68Q135 26 162 26Q197 26 230 60T283 144Q285 150 288 151T303 153H307Q322 153 322 145Q322 142 319 133Q314 117 301 95T267 48T216 6T155 -11Q125 -11 98 4T59 56Q57 64 57 83V101L92 241Q127 382 128 383Q128 385 77 385H26Z",[171,376],{"id":377,"d":378},"MJX-2-TEX-I-1D451","M366 683Q367 683 438 688T511 694Q523 694 523 686Q523 679 450 384T375 83T374 68Q374 26 402 26Q411 27 422 35Q443 55 463 131Q469 151 473 152Q475 153 483 153H487H491Q506 153 506 145Q506 140 503 129Q490 79 473 48T445 8T417 -8Q409 -10 393 -10Q359 -10 336 5T306 36L300 51Q299 52 296 50Q294 48 292 46Q233 -10 172 -10Q117 -10 75 30T33 157Q33 205 53 255T101 341Q148 398 195 420T280 442Q336 442 364 400Q369 394 369 396Q370 400 396 505T424 616Q424 629 417 632T378 637H357Q351 643 351 645T353 664Q358 683 366 683ZM352 326Q329 405 277 405Q242 405 210 374T160 293Q131 214 119 129Q119 126 119 118T118 106Q118 61 136 44T179 26Q233 26 290 98L298 109L352 326Z",[171,380],{"id":381,"d":382},"MJX-2-TEX-N-5F","M0 -62V-25H499V-62H0Z",[171,384],{"id":385,"d":386},"MJX-2-TEX-I-1D463","M173 380Q173 405 154 405Q130 405 104 376T61 287Q60 286 59 284T58 281T56 279T53 278T49 278T41 278H27Q21 284 21 287Q21 294 29 316T53 368T97 419T160 441Q202 441 225 417T249 361Q249 344 246 335Q246 329 231 291T200 202T182 113Q182 86 187 69Q200 26 250 26Q287 26 319 60T369 139T398 222T409 277Q409 300 401 317T383 343T365 361T357 383Q357 405 376 424T417 443Q436 443 451 425T467 367Q467 340 455 284T418 159T347 40T241 -11Q177 -11 139 22Q102 54 102 117Q102 148 110 181T151 298Q173 362 173 380Z",[208,388,389],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,390,391,396,401,406,411,417,423,429,435,443,450,457,464,471,476,481],{"dataMmlNode":215},[208,392,393],{"dataMmlNode":218},[220,394],{"dataC":222,"xLinkHref":395},"#MJX-2-TEX-N-28",[208,397,398],{"dataMmlNode":226,"transform":227},[220,399],{"dataC":230,"xLinkHref":400},"#MJX-2-TEX-I-1D465",[208,402,403],{"dataMmlNode":218,"transform":234},[220,404],{"dataC":237,"xLinkHref":405},"#MJX-2-TEX-N-2212",[208,407,408],{"dataMmlNode":226,"transform":241},[220,409],{"dataC":244,"xLinkHref":410},"#MJX-2-TEX-I-1D45A",[208,412,413],{"dataMmlNode":226,"transform":248},[220,414],{"dataC":415,"xLinkHref":416},"1D452","#MJX-2-TEX-I-1D452",[208,418,420],{"dataMmlNode":226,"transform":419},"translate(3527.4,0)",[220,421],{"dataC":293,"xLinkHref":422},"#MJX-2-TEX-I-1D44E",[208,424,426],{"dataMmlNode":226,"transform":425},"translate(4056.4,0)",[220,427],{"dataC":258,"xLinkHref":428},"#MJX-2-TEX-I-1D45B",[208,430,432],{"dataMmlNode":218,"transform":431},"translate(4656.4,0)",[220,433],{"dataC":265,"xLinkHref":434},"#MJX-2-TEX-N-29",[208,436,438],{"dataMmlNode":269,"dataMjxTexclass":270,"transform":437},"translate(5045.4,0)",[208,439,440],{"dataMmlNode":218},[220,441],{"dataC":276,"xLinkHref":442},"#MJX-2-TEX-N-2F",[208,444,446],{"dataMmlNode":226,"transform":445},"translate(5545.4,0)",[220,447],{"dataC":448,"xLinkHref":449},"1D460","#MJX-2-TEX-I-1D460",[208,451,453],{"dataMmlNode":226,"transform":452},"translate(6014.4,0)",[220,454],{"dataC":455,"xLinkHref":456},"1D461","#MJX-2-TEX-I-1D461",[208,458,460],{"dataMmlNode":226,"transform":459},"translate(6375.4,0)",[220,461],{"dataC":462,"xLinkHref":463},"1D451","#MJX-2-TEX-I-1D451",[208,465,467],{"dataMmlNode":226,"transform":466},"translate(6895.4,0)",[220,468],{"dataC":469,"xLinkHref":470},"5F","#MJX-2-TEX-N-5F",[208,472,474],{"dataMmlNode":226,"transform":473},"translate(7395.4,0)",[220,475],{"dataC":462,"xLinkHref":463},[208,477,479],{"dataMmlNode":226,"transform":478},"translate(7915.4,0)",[220,480],{"dataC":415,"xLinkHref":416},[208,482,484],{"dataMmlNode":226,"transform":483},"translate(8381.4,0)",[220,485],{"dataC":486,"xLinkHref":487},"1D463","#MJX-2-TEX-I-1D463",". Min-Max is great when you know your data has fixed bounds and no extreme outliers. Standardisation is more robust to outliers and is generally preferred for algorithms like logistic regression or SVMs.",[16,490,491],{},"Not all algorithms need feature scaling. Tree-based models (like decision trees and random forests) split on thresholds and don't care about magnitude, so scaling doesn't affect them. It matters most for distance-based and gradient-based algorithms.",[16,493,494,497,498,501,502,505],{},[20,495,496],{},"Step 7 — Split the dataset."," Divide your data into a ",[20,499,500],{},"training set"," (for learning) and a ",[20,503,504],{},"test set"," (for validation). Common splits follow the Pareto principle: 80\u002F20, 70\u002F30, or 60\u002F40.",[16,507,508],{},[163,509],{"alt":510,"src":511},"0.63","https:\u002F\u002Fraw.githubusercontent.com\u002FChinHongTan\u002Fblog\u002Fmain\u002Fpublic\u002Fimages\u002Fuploads\u002F1777275466503-Screenshot_2026-04-27_at_3.37.33_PM.png",[28,513,515],{"id":514},"other-preprocessing-techniques","Other Preprocessing Techniques",[16,517,518],{},"Beyond the core 7 steps, you may also encounter:",[520,521,522,529,535],"ul",{},[523,524,525,528],"li",{},[20,526,527],{},"Data Integration"," — combining data from multiple source systems into a unified set.",[523,530,531,534],{},[20,532,533],{},"Data Transformation"," — converting data from one format to another (e.g., source system format → destination format).",[523,536,537,540],{},[20,538,539],{},"Data Discretization"," — turning continuous values into discrete intervals (e.g., converting exact ages into age groups like \"18–25\", \"26–35\").",[11,542,544],{"id":543},"regression-analysis","Regression Analysis",[16,546,547],{},"Now that our data is clean, let's learn how to make predictions. We'll start with a classic example.",[28,549,551],{"id":550},"the-housing-price-problem","The Housing Price Problem",[16,553,554,555],{},"Imagine we have a dataset of 47 houses in SS17, Petaling Jaya, with their living areas (in square feet) and prices (in RM thousands). The question is simple: ",[20,556,557],{},"given the size of a house, can we predict its price?",[16,559,560,561,564,565,568],{},"This is a ",[20,562,563],{},"supervised learning"," problem. We have input-output pairs and want to learn a mapping between them. Since the output (price) is a continuous value, this is specifically called a ",[20,566,567],{},"regression"," problem.",[79,570,572,575],{"className":571},[72,73],[16,573,574],{},"Regression is a method for finding the relationship between variables so you can predict a continuous outcome from one or more inputs.",[16,576,577],{},[578,579,580,584],"figure",{},[163,581],{"alt":582,"src":583},"0.96","https:\u002F\u002Fraw.githubusercontent.com\u002FChinHongTan\u002Fblog\u002Fmain\u002Fpublic\u002Fimages\u002Fuploads\u002F1777279629723-1N1-K-A43_98pYZ27fnupDA-2048x1418.jpeg",[585,586,587],"figcaption",{},"Linear Regression, Image by Dan White 1000 on Shutterstock",[28,589,591],{"id":590},"notation","Notation",[16,593,594],{},"Before diving in, let's set up our notation:",[520,596,597,656,715,815,842],{},[523,598,599,655],{},[20,600,601],{},[151,602,604],{"className":603,"jax":155},[154],[157,605,610,624],{"style":606,"xmlns":160,"width":607,"height":608,"role":163,"focusable":164,"viewBox":609,"xmlnsXLink":166},"vertical-align: -0.025ex;","3.278ex","2.046ex","0 -893.3 1449.1 904.3",[168,611,612,615,618,621],{},[171,613],{"id":614,"d":178},"MJX-3-TEX-I-1D465",[171,616],{"id":617,"d":174},"MJX-3-TEX-N-28",[171,619],{"id":620,"d":190},"MJX-3-TEX-I-1D456",[171,622],{"id":623,"d":198},"MJX-3-TEX-N-29",[208,625,626],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,627,628],{"dataMmlNode":215},[208,629,631,636],{"dataMmlNode":630},"msup",[208,632,633],{"dataMmlNode":226},[220,634],{"dataC":230,"xLinkHref":635},"#MJX-3-TEX-I-1D465",[208,637,639,644,649],{"dataMmlNode":269,"transform":638,"dataMjxTexclass":270},"translate(605,363) scale(0.707)",[208,640,641],{"dataMmlNode":218},[220,642],{"dataC":222,"xLinkHref":643},"#MJX-3-TEX-N-28",[208,645,646],{"dataMmlNode":226,"transform":227},[220,647],{"dataC":251,"xLinkHref":648},"#MJX-3-TEX-I-1D456",[208,650,652],{"dataMmlNode":218,"transform":651},"translate(734,0)",[220,653],{"dataC":265,"xLinkHref":654},"#MJX-3-TEX-N-29"," = input features (e.g., living area of the i-th house)",[523,657,658,714],{},[20,659,660],{},[151,661,663],{"className":662,"jax":155},[154],[157,664,669,684],{"style":665,"xmlns":160,"width":666,"height":667,"role":163,"focusable":164,"viewBox":668,"xmlnsXLink":166},"vertical-align: -0.464ex;","3.093ex","2.485ex","0 -893.3 1367.1 1098.3",[168,670,671,675,678,681],{},[171,672],{"id":673,"d":674},"MJX-4-TEX-I-1D466","M21 287Q21 301 36 335T84 406T158 442Q199 442 224 419T250 355Q248 336 247 334Q247 331 231 288T198 191T182 105Q182 62 196 45T238 27Q261 27 281 38T312 61T339 94Q339 95 344 114T358 173T377 247Q415 397 419 404Q432 431 462 431Q475 431 483 424T494 412T496 403Q496 390 447 193T391 -23Q363 -106 294 -155T156 -205Q111 -205 77 -183T43 -117Q43 -95 50 -80T69 -58T89 -48T106 -45Q150 -45 150 -87Q150 -107 138 -122T115 -142T102 -147L99 -148Q101 -153 118 -160T152 -167H160Q177 -167 186 -165Q219 -156 247 -127T290 -65T313 -9T321 21L315 17Q309 13 296 6T270 -6Q250 -11 231 -11Q185 -11 150 11T104 82Q103 89 103 113Q103 170 138 262T173 379Q173 380 173 381Q173 390 173 393T169 400T158 404H154Q131 404 112 385T82 344T65 302T57 280Q55 278 41 278H27Q21 284 21 287Z",[171,676],{"id":677,"d":174},"MJX-4-TEX-N-28",[171,679],{"id":680,"d":190},"MJX-4-TEX-I-1D456",[171,682],{"id":683,"d":198},"MJX-4-TEX-N-29",[208,685,686],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,687,688],{"dataMmlNode":215},[208,689,690,696],{"dataMmlNode":630},[208,691,692],{"dataMmlNode":226},[220,693],{"dataC":694,"xLinkHref":695},"1D466","#MJX-4-TEX-I-1D466",[208,697,699,704,709],{"dataMmlNode":269,"transform":698,"dataMjxTexclass":270},"translate(523,363) scale(0.707)",[208,700,701],{"dataMmlNode":218},[220,702],{"dataC":222,"xLinkHref":703},"#MJX-4-TEX-N-28",[208,705,706],{"dataMmlNode":226,"transform":227},[220,707],{"dataC":251,"xLinkHref":708},"#MJX-4-TEX-I-1D456",[208,710,711],{"dataMmlNode":218,"transform":651},[220,712],{"dataC":265,"xLinkHref":713},"#MJX-4-TEX-N-29"," = output\u002Ftarget variable (e.g., price of the i-th house)",[523,716,717,814],{},[20,718,719],{},[151,720,722],{"className":721,"jax":155},[154],[157,723,727,748],{"style":159,"xmlns":160,"width":724,"height":725,"role":163,"focusable":164,"viewBox":726,"xmlnsXLink":166},"9.138ex","2.587ex","0 -893.3 4038.8 1143.3",[168,728,729,732,735,738,741,745],{},[171,730],{"id":731,"d":174},"MJX-5-TEX-N-28",[171,733],{"id":734,"d":178},"MJX-5-TEX-I-1D465",[171,736],{"id":737,"d":190},"MJX-5-TEX-I-1D456",[171,739],{"id":740,"d":198},"MJX-5-TEX-N-29",[171,742],{"id":743,"d":744},"MJX-5-TEX-N-2C","M78 35T78 60T94 103T137 121Q165 121 187 96T210 8Q210 -27 201 -60T180 -117T154 -158T130 -185T117 -194Q113 -194 104 -185T95 -172Q95 -168 106 -156T131 -126T157 -76T173 -3V9L172 8Q170 7 167 6T161 3T152 1T140 0Q113 0 96 17Z",[171,746],{"id":747,"d":674},"MJX-5-TEX-I-1D466",[208,749,750],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,751,752,757,780,787,809],{"dataMmlNode":215},[208,753,754],{"dataMmlNode":218},[220,755],{"dataC":222,"xLinkHref":756},"#MJX-5-TEX-N-28",[208,758,759,764],{"dataMmlNode":630,"transform":227},[208,760,761],{"dataMmlNode":226},[220,762],{"dataC":230,"xLinkHref":763},"#MJX-5-TEX-I-1D465",[208,765,766,770,775],{"dataMmlNode":269,"transform":638,"dataMjxTexclass":270},[208,767,768],{"dataMmlNode":218},[220,769],{"dataC":222,"xLinkHref":756},[208,771,772],{"dataMmlNode":226,"transform":227},[220,773],{"dataC":251,"xLinkHref":774},"#MJX-5-TEX-I-1D456",[208,776,777],{"dataMmlNode":218,"transform":651},[220,778],{"dataC":265,"xLinkHref":779},"#MJX-5-TEX-N-29",[208,781,783],{"dataMmlNode":218,"transform":782},"translate(1838.1,0)",[220,784],{"dataC":785,"xLinkHref":786},"2C","#MJX-5-TEX-N-2C",[208,788,790,795],{"dataMmlNode":630,"transform":789},"translate(2282.7,0)",[208,791,792],{"dataMmlNode":226},[220,793],{"dataC":694,"xLinkHref":794},"#MJX-5-TEX-I-1D466",[208,796,797,801,805],{"dataMmlNode":269,"transform":698,"dataMjxTexclass":270},[208,798,799],{"dataMmlNode":218},[220,800],{"dataC":222,"xLinkHref":756},[208,802,803],{"dataMmlNode":226,"transform":227},[220,804],{"dataC":251,"xLinkHref":774},[208,806,807],{"dataMmlNode":218,"transform":651},[220,808],{"dataC":265,"xLinkHref":779},[208,810,812],{"dataMmlNode":218,"transform":811},"translate(3649.8,0)",[220,813],{"dataC":265,"xLinkHref":779}," = a single training example",[523,816,817,841],{},[20,818,819],{},[151,820,822],{"className":821,"jax":155},[154],[157,823,827,832],{"style":606,"xmlns":160,"width":824,"height":825,"role":163,"focusable":164,"viewBox":826,"xmlnsXLink":166},"1.986ex","1.025ex","0 -442 878 453",[168,828,829],{},[171,830],{"id":831,"d":186},"MJX-6-TEX-I-1D45A",[208,833,834],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,835,836],{"dataMmlNode":215},[208,837,838],{"dataMmlNode":226},[220,839],{"dataC":244,"xLinkHref":840},"#MJX-6-TEX-I-1D45A"," = total number of training examples",[523,843,844,896],{},[20,845,846],{},[151,847,849],{"className":848,"jax":155},[154],[157,850,853,868],{"style":159,"xmlns":160,"width":851,"height":162,"role":163,"focusable":164,"viewBox":852,"xmlnsXLink":166},"4.357ex","0 -750 1926 1000",[168,854,855,859,862,865],{},[171,856],{"id":857,"d":858},"MJX-7-TEX-I-210E","M137 683Q138 683 209 688T282 694Q294 694 294 685Q294 674 258 534Q220 386 220 383Q220 381 227 388Q288 442 357 442Q411 442 444 415T478 336Q478 285 440 178T402 50Q403 36 407 31T422 26Q450 26 474 56T513 138Q516 149 519 151T535 153Q555 153 555 145Q555 144 551 130Q535 71 500 33Q466 -10 419 -10H414Q367 -10 346 17T325 74Q325 90 361 192T398 345Q398 404 354 404H349Q266 404 205 306L198 293L164 158Q132 28 127 16Q114 -11 83 -11Q69 -11 59 -2T48 16Q48 30 121 320L195 616Q195 629 188 632T149 637H128Q122 643 122 645T124 664Q129 683 137 683Z",[171,860],{"id":861,"d":174},"MJX-7-TEX-N-28",[171,863],{"id":864,"d":178},"MJX-7-TEX-I-1D465",[171,866],{"id":867,"d":198},"MJX-7-TEX-N-29",[208,869,870],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,871,872,878,884,890],{"dataMmlNode":215},[208,873,874],{"dataMmlNode":226},[220,875],{"dataC":876,"xLinkHref":877},"210E","#MJX-7-TEX-I-210E",[208,879,881],{"dataMmlNode":218,"transform":880},"translate(576,0)",[220,882],{"dataC":222,"xLinkHref":883},"#MJX-7-TEX-N-28",[208,885,887],{"dataMmlNode":226,"transform":886},"translate(965,0)",[220,888],{"dataC":230,"xLinkHref":889},"#MJX-7-TEX-I-1D465",[208,891,893],{"dataMmlNode":218,"transform":892},"translate(1537,0)",[220,894],{"dataC":265,"xLinkHref":895},"#MJX-7-TEX-N-29"," = our hypothesis function — the predictor we want to learn",[16,898,899,900,943],{},"The goal: learn a function h such that ",[151,901,903],{"className":902,"jax":155},[154],[157,904,905,919],{"style":159,"xmlns":160,"width":851,"height":162,"role":163,"focusable":164,"viewBox":852,"xmlnsXLink":166},[168,906,907,910,913,916],{},[171,908],{"id":909,"d":858},"MJX-8-TEX-I-210E",[171,911],{"id":912,"d":174},"MJX-8-TEX-N-28",[171,914],{"id":915,"d":178},"MJX-8-TEX-I-1D465",[171,917],{"id":918,"d":198},"MJX-8-TEX-N-29",[208,920,921],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,922,923,928,933,938],{"dataMmlNode":215},[208,924,925],{"dataMmlNode":226},[220,926],{"dataC":876,"xLinkHref":927},"#MJX-8-TEX-I-210E",[208,929,930],{"dataMmlNode":218,"transform":880},[220,931],{"dataC":222,"xLinkHref":932},"#MJX-8-TEX-N-28",[208,934,935],{"dataMmlNode":226,"transform":886},[220,936],{"dataC":230,"xLinkHref":937},"#MJX-8-TEX-I-1D465",[208,939,940],{"dataMmlNode":218,"transform":892},[220,941],{"dataC":265,"xLinkHref":942},"#MJX-8-TEX-N-29"," is a good predictor for y.",[28,945,947],{"id":946},"linear-regression","Linear Regression",[16,949,950,951,986,987,1013],{},"The simplest hypothesis is a linear function. If we have two features — living area (",[151,952,954],{"className":953,"jax":155},[154],[157,955,960,965],{"style":956,"xmlns":160,"width":957,"height":958,"role":163,"focusable":164,"viewBox":959,"xmlnsXLink":166},"vertical-align: -0.452ex;","2.652ex","2.149ex","0 -750 1172 950",[168,961,962],{},[171,963],{"id":964,"d":178},"MJX-9-TEX-I-1D465",[208,966,967],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,968,969,974],{"dataMmlNode":215},[208,970,971],{"dataMmlNode":226},[220,972],{"dataC":230,"xLinkHref":973},"#MJX-9-TEX-I-1D465",[208,975,977],{"dataMmlNode":269,"dataMjxTexclass":270,"transform":976},"translate(572,0)",[208,978,979],{"dataMmlNode":218},[980,981,985],"text",{"dataVariant":982,"transform":212,"font-size":983,"font-family":984},"normal","884px","serif","₁",") and number of bedrooms (",[151,988,990],{"className":989,"jax":155},[154],[157,991,992,997],{"style":956,"xmlns":160,"width":957,"height":958,"role":163,"focusable":164,"viewBox":959,"xmlnsXLink":166},[168,993,994],{},[171,995],{"id":996,"d":178},"MJX-10-TEX-I-1D465",[208,998,999],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,1000,1001,1006],{"dataMmlNode":215},[208,1002,1003],{"dataMmlNode":226},[220,1004],{"dataC":230,"xLinkHref":1005},"#MJX-10-TEX-I-1D465",[208,1007,1008],{"dataMmlNode":269,"dataMjxTexclass":270,"transform":976},[208,1009,1010],{"dataMmlNode":218},[980,1011,1012],{"dataVariant":982,"transform":212,"font-size":983,"font-family":984},"₂",") — our model looks like:",[151,1015,1018],{"className":1016,"jax":155,"display":1017},[154],"true",[157,1019,1022,1060],{"style":159,"xmlns":160,"width":1020,"height":162,"role":163,"focusable":164,"viewBox":1021,"xmlnsXLink":166},"23.616ex","0 -750 10438.2 1000",[168,1023,1024,1027,1030,1033,1036,1040,1044,1048,1052,1056],{},[171,1025],{"id":1026,"d":858},"MJX-11-TEX-I-210E",[171,1028],{"id":1029,"d":174},"MJX-11-TEX-N-28",[171,1031],{"id":1032,"d":178},"MJX-11-TEX-I-1D465",[171,1034],{"id":1035,"d":198},"MJX-11-TEX-N-29",[171,1037],{"id":1038,"d":1039},"MJX-11-TEX-N-3D","M56 347Q56 360 70 367H707Q722 359 722 347Q722 336 708 328L390 327H72Q56 332 56 347ZM56 153Q56 168 72 173H708Q722 163 722 153Q722 140 707 133H70Q56 140 56 153Z",[171,1041],{"id":1042,"d":1043},"MJX-11-TEX-I-1D703","M35 200Q35 302 74 415T180 610T319 704Q320 704 327 704T339 705Q393 701 423 656Q462 596 462 495Q462 380 417 261T302 66T168 -10H161Q125 -10 99 10T60 63T41 130T35 200ZM383 566Q383 668 330 668Q294 668 260 623T204 521T170 421T157 371Q206 370 254 370L351 371Q352 372 359 404T375 484T383 566ZM113 132Q113 26 166 26Q181 26 198 36T239 74T287 161T335 307L340 324H145Q145 321 136 286T120 208T113 132Z",[171,1045],{"id":1046,"d":1047},"MJX-11-TEX-N-30","M96 585Q152 666 249 666Q297 666 345 640T423 548Q460 465 460 320Q460 165 417 83Q397 41 362 16T301 -15T250 -22Q224 -22 198 -16T137 16T82 83Q39 165 39 320Q39 494 96 585ZM321 597Q291 629 250 629Q208 629 178 597Q153 571 145 525T137 333Q137 175 145 125T181 46Q209 16 250 16Q290 16 318 46Q347 76 354 130T362 333Q362 478 354 524T321 597Z",[171,1049],{"id":1050,"d":1051},"MJX-11-TEX-N-2B","M56 237T56 250T70 270H369V420L370 570Q380 583 389 583Q402 583 409 568V270H707Q722 262 722 250T707 230H409V-68Q401 -82 391 -82H389H387Q375 -82 369 -68V230H70Q56 237 56 250Z",[171,1053],{"id":1054,"d":1055},"MJX-11-TEX-N-31","M213 578L200 573Q186 568 160 563T102 556H83V602H102Q149 604 189 617T245 641T273 663Q275 666 285 666Q294 666 302 660V361L303 61Q310 54 315 52T339 48T401 46H427V0H416Q395 3 257 3Q121 3 100 0H88V46H114Q136 46 152 46T177 47T193 50T201 52T207 57T213 61V578Z",[171,1057],{"id":1058,"d":1059},"MJX-11-TEX-N-32","M109 429Q82 429 66 447T50 491Q50 562 103 614T235 666Q326 666 387 610T449 465Q449 422 429 383T381 315T301 241Q265 210 201 149L142 93L218 92Q375 92 385 97Q392 99 409 186V189H449V186Q448 183 436 95T421 3V0H50V19V31Q50 38 56 46T86 81Q115 113 136 137Q145 147 170 174T204 211T233 244T261 278T284 308T305 340T320 369T333 401T340 431T343 464Q343 527 309 573T212 619Q179 619 154 602T119 569T109 550Q109 549 114 549Q132 549 151 535T170 489Q170 464 154 447T109 429Z",[208,1061,1062],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,1063,1064,1069,1074,1079,1084,1091,1109,1116,1129,1141,1146,1159],{"dataMmlNode":215},[208,1065,1066],{"dataMmlNode":226},[220,1067],{"dataC":876,"xLinkHref":1068},"#MJX-11-TEX-I-210E",[208,1070,1071],{"dataMmlNode":218,"transform":880},[220,1072],{"dataC":222,"xLinkHref":1073},"#MJX-11-TEX-N-28",[208,1075,1076],{"dataMmlNode":226,"transform":886},[220,1077],{"dataC":230,"xLinkHref":1078},"#MJX-11-TEX-I-1D465",[208,1080,1081],{"dataMmlNode":218,"transform":892},[220,1082],{"dataC":265,"xLinkHref":1083},"#MJX-11-TEX-N-29",[208,1085,1087],{"dataMmlNode":218,"transform":1086},"translate(2203.8,0)",[220,1088],{"dataC":1089,"xLinkHref":1090},"3D","#MJX-11-TEX-N-3D",[208,1092,1095,1101],{"dataMmlNode":1093,"transform":1094},"msub","translate(3259.6,0)",[208,1096,1097],{"dataMmlNode":226},[220,1098],{"dataC":1099,"xLinkHref":1100},"1D703","#MJX-11-TEX-I-1D703",[208,1102,1105],{"dataMmlNode":1103,"transform":1104},"mn","translate(502,-150) scale(0.707)",[220,1106],{"dataC":1107,"xLinkHref":1108},"30","#MJX-11-TEX-N-30",[208,1110,1112],{"dataMmlNode":218,"transform":1111},"translate(4387.3,0)",[220,1113],{"dataC":1114,"xLinkHref":1115},"2B","#MJX-11-TEX-N-2B",[208,1117,1119,1123],{"dataMmlNode":1093,"transform":1118},"translate(5387.6,0)",[208,1120,1121],{"dataMmlNode":226},[220,1122],{"dataC":1099,"xLinkHref":1100},[208,1124,1125],{"dataMmlNode":1103,"transform":1104},[220,1126],{"dataC":1127,"xLinkHref":1128},"31","#MJX-11-TEX-N-31",[208,1130,1132,1136],{"dataMmlNode":1093,"transform":1131},"translate(6293.1,0)",[208,1133,1134],{"dataMmlNode":226},[220,1135],{"dataC":230,"xLinkHref":1078},[208,1137,1139],{"dataMmlNode":1103,"transform":1138},"translate(605,-150) scale(0.707)",[220,1140],{"dataC":1127,"xLinkHref":1128},[208,1142,1144],{"dataMmlNode":218,"transform":1143},"translate(7523.9,0)",[220,1145],{"dataC":1114,"xLinkHref":1115},[208,1147,1149,1153],{"dataMmlNode":1093,"transform":1148},"translate(8524.1,0)",[208,1150,1151],{"dataMmlNode":226},[220,1152],{"dataC":1099,"xLinkHref":1100},[208,1154,1155],{"dataMmlNode":1103,"transform":1104},[220,1156],{"dataC":1157,"xLinkHref":1158},"32","#MJX-11-TEX-N-32",[208,1160,1162,1166],{"dataMmlNode":1093,"transform":1161},"translate(9429.7,0)",[208,1163,1164],{"dataMmlNode":226},[220,1165],{"dataC":230,"xLinkHref":1078},[208,1167,1168],{"dataMmlNode":1103,"transform":1138},[220,1169],{"dataC":1157,"xLinkHref":1158},[16,1171,1172,1173,1196,1197,1200,1201,1204,1205,1224],{},"The ",[151,1174,1176],{"className":1175,"jax":155},[154],[157,1177,1182,1187],{"style":1178,"xmlns":160,"width":1179,"height":1180,"role":163,"focusable":164,"viewBox":1181,"xmlnsXLink":166},"vertical-align: -0.023ex;","1.061ex","1.618ex","0 -705 469 715",[168,1183,1184],{},[171,1185],{"id":1186,"d":1043},"MJX-12-TEX-I-1D703",[208,1188,1189],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,1190,1191],{"dataMmlNode":215},[208,1192,1193],{"dataMmlNode":226},[220,1194],{"dataC":1099,"xLinkHref":1195},"#MJX-12-TEX-I-1D703"," values are called ",[20,1198,1199],{},"parameters"," (or ",[20,1202,1203],{},"weights","). Our job is to find the best ",[151,1206,1208],{"className":1207,"jax":155},[154],[157,1209,1210,1215],{"style":1178,"xmlns":160,"width":1179,"height":1180,"role":163,"focusable":164,"viewBox":1181,"xmlnsXLink":166},[168,1211,1212],{},[171,1213],{"id":1214,"d":1043},"MJX-13-TEX-I-1D703",[208,1216,1217],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,1218,1219],{"dataMmlNode":215},[208,1220,1221],{"dataMmlNode":226},[220,1222],{"dataC":1099,"xLinkHref":1223},"#MJX-13-TEX-I-1D703"," values. But how?",[28,1226,1228],{"id":1227},"the-cost-function","The Cost Function",[16,1230,1231,1232,1235],{},"We define a ",[20,1233,1234],{},"least-squares cost function"," that measures how far off our predictions are from the actual values:",[151,1237,1239],{"className":1238,"jax":155,"display":1017},[154],[157,1240,1245,1288],{"style":1241,"xmlns":160,"width":1242,"height":1243,"role":163,"focusable":164,"viewBox":1244,"xmlnsXLink":166},"vertical-align: -2.697ex;","27.99ex","5.733ex","0 -1342 12371.4 2534.2",[168,1246,1247,1251,1254,1257,1260,1263,1266,1269,1273,1276,1279,1282,1285],{},[171,1248],{"id":1249,"d":1250},"MJX-14-TEX-I-1D43D","M447 625Q447 637 354 637H329Q323 642 323 645T325 664Q329 677 335 683H352Q393 681 498 681Q541 681 568 681T605 682T619 682Q633 682 633 672Q633 670 630 658Q626 642 623 640T604 637Q552 637 545 623Q541 610 483 376Q420 128 419 127Q397 64 333 21T195 -22Q137 -22 97 8T57 88Q57 130 80 152T132 174Q177 174 182 130Q182 98 164 80T123 56Q115 54 115 53T122 44Q148 15 197 15Q235 15 271 47T324 130Q328 142 387 380T447 625Z",[171,1252],{"id":1253,"d":174},"MJX-14-TEX-N-28",[171,1255],{"id":1256,"d":1043},"MJX-14-TEX-I-1D703",[171,1258],{"id":1259,"d":198},"MJX-14-TEX-N-29",[171,1261],{"id":1262,"d":1039},"MJX-14-TEX-N-3D",[171,1264],{"id":1265,"d":1055},"MJX-14-TEX-N-31",[171,1267],{"id":1268,"d":1059},"MJX-14-TEX-N-32",[171,1270],{"id":1271,"d":1272},"MJX-14-TEX-LO-2211","M60 948Q63 950 665 950H1267L1325 815Q1384 677 1388 669H1348L1341 683Q1320 724 1285 761Q1235 809 1174 838T1033 881T882 898T699 902H574H543H251L259 891Q722 258 724 252Q725 250 724 246Q721 243 460 -56L196 -356Q196 -357 407 -357Q459 -357 548 -357T676 -358Q812 -358 896 -353T1063 -332T1204 -283T1307 -196Q1328 -170 1348 -124H1388Q1388 -125 1381 -145T1356 -210T1325 -294L1267 -449L666 -450Q64 -450 61 -448Q55 -446 55 -439Q55 -437 57 -433L590 177Q590 178 557 222T452 366T322 544L56 909L55 924Q55 945 60 948Z",[171,1274],{"id":1275,"d":190},"MJX-14-TEX-I-1D456",[171,1277],{"id":1278,"d":858},"MJX-14-TEX-I-210E",[171,1280],{"id":1281,"d":178},"MJX-14-TEX-I-1D465",[171,1283],{"id":1284,"d":182},"MJX-14-TEX-N-2212",[171,1286],{"id":1287,"d":674},"MJX-14-TEX-I-1D466",[208,1289,1290],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,1291,1292,1298,1304,1310,1316,1322,1345,1361,1366,1372,1377,1400,1405,1411,1434],{"dataMmlNode":215},[208,1293,1294],{"dataMmlNode":226},[220,1295],{"dataC":1296,"xLinkHref":1297},"1D43D","#MJX-14-TEX-I-1D43D",[208,1299,1301],{"dataMmlNode":218,"transform":1300},"translate(633,0)",[220,1302],{"dataC":222,"xLinkHref":1303},"#MJX-14-TEX-N-28",[208,1305,1307],{"dataMmlNode":226,"transform":1306},"translate(1022,0)",[220,1308],{"dataC":1099,"xLinkHref":1309},"#MJX-14-TEX-I-1D703",[208,1311,1313],{"dataMmlNode":218,"transform":1312},"translate(1491,0)",[220,1314],{"dataC":265,"xLinkHref":1315},"#MJX-14-TEX-N-29",[208,1317,1319],{"dataMmlNode":218,"transform":1318},"translate(2157.8,0)",[220,1320],{"dataC":1089,"xLinkHref":1321},"#MJX-14-TEX-N-3D",[208,1323,1326,1332,1338],{"dataMmlNode":1324,"transform":1325},"mfrac","translate(3213.6,0)",[208,1327,1329],{"dataMmlNode":1103,"transform":1328},"translate(220,676)",[220,1330],{"dataC":1127,"xLinkHref":1331},"#MJX-14-TEX-N-31",[208,1333,1335],{"dataMmlNode":1103,"transform":1334},"translate(220,-686)",[220,1336],{"dataC":1157,"xLinkHref":1337},"#MJX-14-TEX-N-32",[1339,1340],"rect",{"width":1341,"height":1342,"x":1343,"y":1344},700,60,"120","220",[208,1346,1349,1355],{"dataMmlNode":1347,"transform":1348},"munder","translate(4320.2,0)",[208,1350,1351],{"dataMmlNode":218},[220,1352],{"dataC":1353,"xLinkHref":1354},"2211","#MJX-14-TEX-LO-2211",[208,1356,1358],{"dataMmlNode":226,"transform":1357},"translate(600,-1084.4) scale(0.707)",[220,1359],{"dataC":251,"xLinkHref":1360},"#MJX-14-TEX-I-1D456",[208,1362,1364],{"dataMmlNode":218,"transform":1363},"translate(5764.2,0)",[220,1365],{"dataC":222,"xLinkHref":1303},[208,1367,1369],{"dataMmlNode":226,"transform":1368},"translate(6153.2,0)",[220,1370],{"dataC":876,"xLinkHref":1371},"#MJX-14-TEX-I-210E",[208,1373,1375],{"dataMmlNode":218,"transform":1374},"translate(6729.2,0)",[220,1376],{"dataC":222,"xLinkHref":1303},[208,1378,1380,1385],{"dataMmlNode":630,"transform":1379},"translate(7118.2,0)",[208,1381,1382],{"dataMmlNode":226},[220,1383],{"dataC":230,"xLinkHref":1384},"#MJX-14-TEX-I-1D465",[208,1386,1388,1392,1396],{"dataMmlNode":269,"transform":1387,"dataMjxTexclass":270},"translate(605,413) scale(0.707)",[208,1389,1390],{"dataMmlNode":218},[220,1391],{"dataC":222,"xLinkHref":1303},[208,1393,1394],{"dataMmlNode":226,"transform":227},[220,1395],{"dataC":251,"xLinkHref":1360},[208,1397,1398],{"dataMmlNode":218,"transform":651},[220,1399],{"dataC":265,"xLinkHref":1315},[208,1401,1403],{"dataMmlNode":218,"transform":1402},"translate(8567.3,0)",[220,1404],{"dataC":265,"xLinkHref":1315},[208,1406,1408],{"dataMmlNode":218,"transform":1407},"translate(9178.5,0)",[220,1409],{"dataC":237,"xLinkHref":1410},"#MJX-14-TEX-N-2212",[208,1412,1414,1419],{"dataMmlNode":630,"transform":1413},"translate(10178.7,0)",[208,1415,1416],{"dataMmlNode":226},[220,1417],{"dataC":694,"xLinkHref":1418},"#MJX-14-TEX-I-1D466",[208,1420,1422,1426,1430],{"dataMmlNode":269,"transform":1421,"dataMjxTexclass":270},"translate(523,413) scale(0.707)",[208,1423,1424],{"dataMmlNode":218},[220,1425],{"dataC":222,"xLinkHref":1303},[208,1427,1428],{"dataMmlNode":226,"transform":227},[220,1429],{"dataC":251,"xLinkHref":1360},[208,1431,1432],{"dataMmlNode":218,"transform":651},[220,1433],{"dataC":265,"xLinkHref":1315},[208,1435,1437,1441],{"dataMmlNode":630,"transform":1436},"translate(11545.8,0)",[208,1438,1439],{"dataMmlNode":218},[220,1440],{"dataC":265,"xLinkHref":1315},[208,1442,1444],{"dataMmlNode":1103,"transform":1443},"translate(422,413) scale(0.707)",[220,1445],{"dataC":1157,"xLinkHref":1337},[16,1447,1448,1449,1494,1495,1498,1499,1542],{},"The smaller ",[151,1450,1452],{"className":1451,"jax":155},[154],[157,1453,1456,1470],{"style":159,"xmlns":160,"width":1454,"height":162,"role":163,"focusable":164,"viewBox":1455,"xmlnsXLink":166},"4.253ex","0 -750 1880 1000",[168,1457,1458,1461,1464,1467],{},[171,1459],{"id":1460,"d":1250},"MJX-15-TEX-I-1D43D",[171,1462],{"id":1463,"d":174},"MJX-15-TEX-N-28",[171,1465],{"id":1466,"d":1043},"MJX-15-TEX-I-1D703",[171,1468],{"id":1469,"d":198},"MJX-15-TEX-N-29",[208,1471,1472],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,1473,1474,1479,1484,1489],{"dataMmlNode":215},[208,1475,1476],{"dataMmlNode":226},[220,1477],{"dataC":1296,"xLinkHref":1478},"#MJX-15-TEX-I-1D43D",[208,1480,1481],{"dataMmlNode":218,"transform":1300},[220,1482],{"dataC":222,"xLinkHref":1483},"#MJX-15-TEX-N-28",[208,1485,1486],{"dataMmlNode":226,"transform":1306},[220,1487],{"dataC":1099,"xLinkHref":1488},"#MJX-15-TEX-I-1D703",[208,1490,1491],{"dataMmlNode":218,"transform":1312},[220,1492],{"dataC":265,"xLinkHref":1493},"#MJX-15-TEX-N-29"," is, the better our model fits the data. So we want to ",[20,1496,1497],{},"minimise"," ",[151,1500,1502],{"className":1501,"jax":155},[154],[157,1503,1504,1518],{"style":159,"xmlns":160,"width":1454,"height":162,"role":163,"focusable":164,"viewBox":1455,"xmlnsXLink":166},[168,1505,1506,1509,1512,1515],{},[171,1507],{"id":1508,"d":1250},"MJX-16-TEX-I-1D43D",[171,1510],{"id":1511,"d":174},"MJX-16-TEX-N-28",[171,1513],{"id":1514,"d":1043},"MJX-16-TEX-I-1D703",[171,1516],{"id":1517,"d":198},"MJX-16-TEX-N-29",[208,1519,1520],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,1521,1522,1527,1532,1537],{"dataMmlNode":215},[208,1523,1524],{"dataMmlNode":226},[220,1525],{"dataC":1296,"xLinkHref":1526},"#MJX-16-TEX-I-1D43D",[208,1528,1529],{"dataMmlNode":218,"transform":1300},[220,1530],{"dataC":222,"xLinkHref":1531},"#MJX-16-TEX-N-28",[208,1533,1534],{"dataMmlNode":226,"transform":1306},[220,1535],{"dataC":1099,"xLinkHref":1536},"#MJX-16-TEX-I-1D703",[208,1538,1539],{"dataMmlNode":218,"transform":1312},[220,1540],{"dataC":265,"xLinkHref":1541},"#MJX-16-TEX-N-29",".",[69,1544,1546,1549],{"className":1545},[72,73],[75,1547,1548],{},"Recap Cost Function",[79,1550,1552],{"className":1551},[82],[16,1553,1554,1555,1562,1563,1566],{},"You can learn more about cost function ",[1556,1557,1561],"a",{"href":1558,"rel":1559},"https:\u002F\u002Fblog.chinono.dev\u002Fblog\u002FIntroduction-to-Machine-Learning#the-lms-least-mean-squares-algorithm",[1560],"nofollow","here",". The core idea remains pretty much the same, only difference is that this formula writes the cost as the sum of all errors, which corresponds to ",[20,1564,1565],{},"batch gradient descent"," where you accumulate the total error across every example before making one update. Batch is more stable per step, but each step is expensive because you have to loop through the entire dataset.",[28,1568,1570],{"id":1569},"gradient-descent-finding-the-best-parameters","Gradient Descent — Finding the Best Parameters",[16,1572,1573],{},"Gradient descent is an iterative optimization algorithm. Here's the intuition:",[1575,1576,1577,1599,1645,1712],"ol",{},[523,1578,1579,1580,1542],{},"Start with a random guess for ",[151,1581,1583],{"className":1582,"jax":155},[154],[157,1584,1585,1590],{"style":1178,"xmlns":160,"width":1179,"height":1180,"role":163,"focusable":164,"viewBox":1181,"xmlnsXLink":166},[168,1586,1587],{},[171,1588],{"id":1589,"d":1043},"MJX-17-TEX-I-1D703",[208,1591,1592],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,1593,1594],{"dataMmlNode":215},[208,1595,1596],{"dataMmlNode":226},[220,1597],{"dataC":1099,"xLinkHref":1598},"#MJX-17-TEX-I-1D703",[523,1600,1601,1602,1542],{},"Compute the cost ",[151,1603,1605],{"className":1604,"jax":155},[154],[157,1606,1607,1621],{"style":159,"xmlns":160,"width":1454,"height":162,"role":163,"focusable":164,"viewBox":1455,"xmlnsXLink":166},[168,1608,1609,1612,1615,1618],{},[171,1610],{"id":1611,"d":1250},"MJX-18-TEX-I-1D43D",[171,1613],{"id":1614,"d":174},"MJX-18-TEX-N-28",[171,1616],{"id":1617,"d":1043},"MJX-18-TEX-I-1D703",[171,1619],{"id":1620,"d":198},"MJX-18-TEX-N-29",[208,1622,1623],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,1624,1625,1630,1635,1640],{"dataMmlNode":215},[208,1626,1627],{"dataMmlNode":226},[220,1628],{"dataC":1296,"xLinkHref":1629},"#MJX-18-TEX-I-1D43D",[208,1631,1632],{"dataMmlNode":218,"transform":1300},[220,1633],{"dataC":222,"xLinkHref":1634},"#MJX-18-TEX-N-28",[208,1636,1637],{"dataMmlNode":226,"transform":1306},[220,1638],{"dataC":1099,"xLinkHref":1639},"#MJX-18-TEX-I-1D703",[208,1641,1642],{"dataMmlNode":218,"transform":1312},[220,1643],{"dataC":265,"xLinkHref":1644},"#MJX-18-TEX-N-29",[523,1646,1647,1648,1667,1668,1711],{},"Adjust ",[151,1649,1651],{"className":1650,"jax":155},[154],[157,1652,1653,1658],{"style":1178,"xmlns":160,"width":1179,"height":1180,"role":163,"focusable":164,"viewBox":1181,"xmlnsXLink":166},[168,1654,1655],{},[171,1656],{"id":1657,"d":1043},"MJX-19-TEX-I-1D703",[208,1659,1660],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,1661,1662],{"dataMmlNode":215},[208,1663,1664],{"dataMmlNode":226},[220,1665],{"dataC":1099,"xLinkHref":1666},"#MJX-19-TEX-I-1D703"," in the direction that reduces ",[151,1669,1671],{"className":1670,"jax":155},[154],[157,1672,1673,1687],{"style":159,"xmlns":160,"width":1454,"height":162,"role":163,"focusable":164,"viewBox":1455,"xmlnsXLink":166},[168,1674,1675,1678,1681,1684],{},[171,1676],{"id":1677,"d":1250},"MJX-20-TEX-I-1D43D",[171,1679],{"id":1680,"d":174},"MJX-20-TEX-N-28",[171,1682],{"id":1683,"d":1043},"MJX-20-TEX-I-1D703",[171,1685],{"id":1686,"d":198},"MJX-20-TEX-N-29",[208,1688,1689],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,1690,1691,1696,1701,1706],{"dataMmlNode":215},[208,1692,1693],{"dataMmlNode":226},[220,1694],{"dataC":1296,"xLinkHref":1695},"#MJX-20-TEX-I-1D43D",[208,1697,1698],{"dataMmlNode":218,"transform":1300},[220,1699],{"dataC":222,"xLinkHref":1700},"#MJX-20-TEX-N-28",[208,1702,1703],{"dataMmlNode":226,"transform":1306},[220,1704],{"dataC":1099,"xLinkHref":1705},"#MJX-20-TEX-I-1D703",[208,1707,1708],{"dataMmlNode":218,"transform":1312},[220,1709],{"dataC":265,"xLinkHref":1710},"#MJX-20-TEX-N-29"," the most (the direction of steepest descent).",[523,1713,1714],{},"Repeat until convergence.\nThe update rule is:",[151,1716,1718],{"className":1717,"jax":155,"display":1017},[154],[157,1719,1724,1760],{"style":1720,"xmlns":160,"width":1721,"height":1722,"role":163,"focusable":164,"viewBox":1723,"xmlnsXLink":166},"vertical-align: -2.218ex;","18.205ex","5.521ex","0 -1460 8046.7 2440.2",[168,1725,1726,1729,1733,1737,1740,1743,1747,1751,1754,1757],{},[171,1727],{"id":1728,"d":1043},"MJX-21-TEX-I-1D703",[171,1730],{"id":1731,"d":1732},"MJX-21-TEX-I-1D457","M297 596Q297 627 318 644T361 661Q378 661 389 651T403 623Q403 595 384 576T340 557Q322 557 310 567T297 596ZM288 376Q288 405 262 405Q240 405 220 393T185 362T161 325T144 293L137 279Q135 278 121 278H107Q101 284 101 286T105 299Q126 348 164 391T252 441Q253 441 260 441T272 442Q296 441 316 432Q341 418 354 401T367 348V332L318 133Q267 -67 264 -75Q246 -125 194 -164T75 -204Q25 -204 7 -183T-12 -137Q-12 -110 7 -91T53 -71Q70 -71 82 -81T95 -112Q95 -148 63 -167Q69 -168 77 -168Q111 -168 139 -140T182 -74L193 -32Q204 11 219 72T251 197T278 308T289 365Q289 372 288 376Z",[171,1734],{"id":1735,"d":1736},"MJX-21-TEX-N-3A","M78 370Q78 394 95 412T138 430Q162 430 180 414T199 371Q199 346 182 328T139 310T96 327T78 370ZM78 60Q78 84 95 102T138 120Q162 120 180 104T199 61Q199 36 182 18T139 0T96 17T78 60Z",[171,1738],{"id":1739,"d":1039},"MJX-21-TEX-N-3D",[171,1741],{"id":1742,"d":182},"MJX-21-TEX-N-2212",[171,1744],{"id":1745,"d":1746},"MJX-21-TEX-I-1D6FC","M34 156Q34 270 120 356T309 442Q379 442 421 402T478 304Q484 275 485 237V208Q534 282 560 374Q564 388 566 390T582 393Q603 393 603 385Q603 376 594 346T558 261T497 161L486 147L487 123Q489 67 495 47T514 26Q528 28 540 37T557 60Q559 67 562 68T577 70Q597 70 597 62Q597 56 591 43Q579 19 556 5T512 -10H505Q438 -10 414 62L411 69L400 61Q390 53 370 41T325 18T267 -2T203 -11Q124 -11 79 39T34 156ZM208 26Q257 26 306 47T379 90L403 112Q401 255 396 290Q382 405 304 405Q235 405 183 332Q156 292 139 224T121 120Q121 71 146 49T208 26Z",[171,1748],{"id":1749,"d":1750},"MJX-21-TEX-I-1D715","M202 508Q179 508 169 520T158 547Q158 557 164 577T185 624T230 675T301 710L333 715H345Q378 715 384 714Q447 703 489 661T549 568T566 457Q566 362 519 240T402 53Q321 -22 223 -22Q123 -22 73 56Q42 102 42 148V159Q42 276 129 370T322 465Q383 465 414 434T455 367L458 378Q478 461 478 515Q478 603 437 639T344 676Q266 676 223 612Q264 606 264 572Q264 547 246 528T202 508ZM430 306Q430 372 401 400T333 428Q270 428 222 382Q197 354 183 323T150 221Q132 149 132 116Q132 21 232 21Q244 21 250 22Q327 35 374 112Q389 137 409 196T430 306Z",[171,1752],{"id":1753,"d":1250},"MJX-21-TEX-I-1D43D",[171,1755],{"id":1756,"d":174},"MJX-21-TEX-N-28",[171,1758],{"id":1759,"d":198},"MJX-21-TEX-N-29",[208,1761,1762],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,1763,1764,1777,1792,1803,1809,1816],{"dataMmlNode":215},[208,1765,1766,1771],{"dataMmlNode":1093},[208,1767,1768],{"dataMmlNode":226},[220,1769],{"dataC":1099,"xLinkHref":1770},"#MJX-21-TEX-I-1D703",[208,1772,1773],{"dataMmlNode":226,"transform":1104},[220,1774],{"dataC":1775,"xLinkHref":1776},"1D457","#MJX-21-TEX-I-1D457",[208,1778,1780,1786],{"dataMmlNode":218,"transform":1779},"translate(1121.1,0)",[208,1781,1782],{"dataMmlNode":980},[220,1783],{"dataC":1784,"xLinkHref":1785},"3A","#MJX-21-TEX-N-3A",[208,1787,1789],{"dataMmlNode":980,"transform":1788},"translate(278,0)",[220,1790],{"dataC":1089,"xLinkHref":1791},"#MJX-21-TEX-N-3D",[208,1793,1795,1799],{"dataMmlNode":1093,"transform":1794},"translate(2454.9,0)",[208,1796,1797],{"dataMmlNode":226},[220,1798],{"dataC":1099,"xLinkHref":1770},[208,1800,1801],{"dataMmlNode":226,"transform":1104},[220,1802],{"dataC":1775,"xLinkHref":1776},[208,1804,1806],{"dataMmlNode":218,"transform":1805},"translate(3520.4,0)",[220,1807],{"dataC":237,"xLinkHref":1808},"#MJX-21-TEX-N-2212",[208,1810,1812],{"dataMmlNode":226,"transform":1811},"translate(4520.7,0)",[220,1813],{"dataC":1814,"xLinkHref":1815},"1D6FC","#MJX-21-TEX-I-1D6FC",[208,1817,1819,1852,1869],{"dataMmlNode":1324,"transform":1818},"translate(5160.7,0)",[208,1820,1823,1829,1835,1841,1846],{"dataMmlNode":1821,"transform":1822},"mrow","translate(220,710)",[208,1824,1825],{"dataMmlNode":226},[220,1826],{"dataC":1827,"xLinkHref":1828},"1D715","#MJX-21-TEX-I-1D715",[208,1830,1832],{"dataMmlNode":226,"transform":1831},"translate(566,0)",[220,1833],{"dataC":1296,"xLinkHref":1834},"#MJX-21-TEX-I-1D43D",[208,1836,1838],{"dataMmlNode":218,"transform":1837},"translate(1199,0)",[220,1839],{"dataC":222,"xLinkHref":1840},"#MJX-21-TEX-N-28",[208,1842,1844],{"dataMmlNode":226,"transform":1843},"translate(1588,0)",[220,1845],{"dataC":1099,"xLinkHref":1770},[208,1847,1849],{"dataMmlNode":218,"transform":1848},"translate(2057,0)",[220,1850],{"dataC":265,"xLinkHref":1851},"#MJX-21-TEX-N-29",[208,1853,1855,1859],{"dataMmlNode":1821,"transform":1854},"translate(738.3,-686)",[208,1856,1857],{"dataMmlNode":226},[220,1858],{"dataC":1827,"xLinkHref":1828},[208,1860,1861,1865],{"dataMmlNode":1093,"transform":1831},[208,1862,1863],{"dataMmlNode":226},[220,1864],{"dataC":1099,"xLinkHref":1770},[208,1866,1867],{"dataMmlNode":226,"transform":1104},[220,1868],{"dataC":1775,"xLinkHref":1776},[1339,1870],{"width":1871,"height":1342,"x":1343,"y":1344},2646,[69,1873,1875,1878],{"className":1874},[72,73],[75,1876,1877],{},"Understanding the formula",[79,1879,1881,1917],{"className":1880},[82],[16,1882,1883,1916],{},[151,1884,1886],{"className":1885,"jax":155},[154],[157,1887,1892,1900],{"style":1888,"xmlns":160,"width":1889,"height":1890,"role":163,"focusable":164,"viewBox":1891,"xmlnsXLink":166},"vertical-align: -0.186ex;","2.389ex","1.505ex","0 -583 1056 665",[168,1893,1894,1897],{},[171,1895],{"id":1896,"d":1736},"MJX-22-TEX-N-3A",[171,1898],{"id":1899,"d":1039},"MJX-22-TEX-N-3D",[208,1901,1902],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,1903,1904],{"dataMmlNode":215},[208,1905,1906,1911],{"dataMmlNode":218},[208,1907,1908],{"dataMmlNode":980},[220,1909],{"dataC":1784,"xLinkHref":1910},"#MJX-22-TEX-N-3A",[208,1912,1913],{"dataMmlNode":980,"transform":1788},[220,1914],{"dataC":1089,"xLinkHref":1915},"#MJX-22-TEX-N-3D"," means \"assign.\" It computes the expression on the right using the current values, then updates the left side with the result.",[16,1918,1919,1920,1925],{},"This is gradient descent in its simplest form, adjust the weight by stepping in the opposite direction of the slope, using a small learning rate so we don't overshoot. If you want to learn more, check out ",[1556,1921,1924],{"href":1922,"rel":1923},"https:\u002F\u002Fblog.chinono.dev\u002Fblog\u002FArtificial-Neural-Networks-and-Backpropagation#updating-output-layer-weights",[1560],"this post",", but just note that it's the same update rule, just applied in a deeper context. Linear regression uses the simplest form of this update, back-propagation extends it to networks with multiple layers.",[16,1927,1928,1929,1952,1953,1956],{},"Here, ",[20,1930,1931],{},[151,1932,1934],{"className":1933,"jax":155},[154],[157,1935,1938,1943],{"style":606,"xmlns":160,"width":1936,"height":825,"role":163,"focusable":164,"viewBox":1937,"xmlnsXLink":166},"1.448ex","0 -442 640 453",[168,1939,1940],{},[171,1941],{"id":1942,"d":1746},"MJX-23-TEX-I-1D6FC",[208,1944,1945],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,1946,1947],{"dataMmlNode":215},[208,1948,1949],{"dataMmlNode":226},[220,1950],{"dataC":1814,"xLinkHref":1951},"#MJX-23-TEX-I-1D6FC"," is the ",[20,1954,1955],{},"learning rate"," — it controls how big each step is. Too large and you overshoot; too small and convergence is painfully slow.",[16,1958,1959],{},"Working out the partial derivative, the update becomes:",[151,1961,1963],{"className":1962,"jax":155,"display":1017},[154],[157,1964,1969,2010],{"style":1965,"xmlns":160,"width":1966,"height":1967,"role":163,"focusable":164,"viewBox":1968,"xmlnsXLink":166},"vertical-align: -0.991ex;","28.915ex","3.391ex","0 -1060.7 12780.3 1498.8",[168,1970,1971,1974,1977,1980,1983,1986,1989,1992,1995,1998,2001,2004,2007],{},[171,1972],{"id":1973,"d":1043},"MJX-24-TEX-I-1D703",[171,1975],{"id":1976,"d":1732},"MJX-24-TEX-I-1D457",[171,1978],{"id":1979,"d":1736},"MJX-24-TEX-N-3A",[171,1981],{"id":1982,"d":1039},"MJX-24-TEX-N-3D",[171,1984],{"id":1985,"d":1051},"MJX-24-TEX-N-2B",[171,1987],{"id":1988,"d":1746},"MJX-24-TEX-I-1D6FC",[171,1990],{"id":1991,"d":174},"MJX-24-TEX-N-28",[171,1993],{"id":1994,"d":674},"MJX-24-TEX-I-1D466",[171,1996],{"id":1997,"d":190},"MJX-24-TEX-I-1D456",[171,1999],{"id":2000,"d":198},"MJX-24-TEX-N-29",[171,2002],{"id":2003,"d":182},"MJX-24-TEX-N-2212",[171,2005],{"id":2006,"d":858},"MJX-24-TEX-I-210E",[171,2008],{"id":2009,"d":178},"MJX-24-TEX-I-1D465",[208,2011,2012],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,2013,2014,2026,2038,2048,2053,2058,2063,2087,2093,2099,2104,2126,2131,2136],{"dataMmlNode":215},[208,2015,2016,2021],{"dataMmlNode":1093},[208,2017,2018],{"dataMmlNode":226},[220,2019],{"dataC":1099,"xLinkHref":2020},"#MJX-24-TEX-I-1D703",[208,2022,2023],{"dataMmlNode":226,"transform":1104},[220,2024],{"dataC":1775,"xLinkHref":2025},"#MJX-24-TEX-I-1D457",[208,2027,2028,2033],{"dataMmlNode":218,"transform":1779},[208,2029,2030],{"dataMmlNode":980},[220,2031],{"dataC":1784,"xLinkHref":2032},"#MJX-24-TEX-N-3A",[208,2034,2035],{"dataMmlNode":980,"transform":1788},[220,2036],{"dataC":1089,"xLinkHref":2037},"#MJX-24-TEX-N-3D",[208,2039,2040,2044],{"dataMmlNode":1093,"transform":1794},[208,2041,2042],{"dataMmlNode":226},[220,2043],{"dataC":1099,"xLinkHref":2020},[208,2045,2046],{"dataMmlNode":226,"transform":1104},[220,2047],{"dataC":1775,"xLinkHref":2025},[208,2049,2050],{"dataMmlNode":218,"transform":1805},[220,2051],{"dataC":1114,"xLinkHref":2052},"#MJX-24-TEX-N-2B",[208,2054,2055],{"dataMmlNode":226,"transform":1811},[220,2056],{"dataC":1814,"xLinkHref":2057},"#MJX-24-TEX-I-1D6FC",[208,2059,2060],{"dataMmlNode":218,"transform":1818},[220,2061],{"dataC":222,"xLinkHref":2062},"#MJX-24-TEX-N-28",[208,2064,2066,2071],{"dataMmlNode":630,"transform":2065},"translate(5549.7,0)",[208,2067,2068],{"dataMmlNode":226},[220,2069],{"dataC":694,"xLinkHref":2070},"#MJX-24-TEX-I-1D466",[208,2072,2073,2077,2082],{"dataMmlNode":269,"transform":1421,"dataMjxTexclass":270},[208,2074,2075],{"dataMmlNode":218},[220,2076],{"dataC":222,"xLinkHref":2062},[208,2078,2079],{"dataMmlNode":226,"transform":227},[220,2080],{"dataC":251,"xLinkHref":2081},"#MJX-24-TEX-I-1D456",[208,2083,2084],{"dataMmlNode":218,"transform":651},[220,2085],{"dataC":265,"xLinkHref":2086},"#MJX-24-TEX-N-29",[208,2088,2090],{"dataMmlNode":218,"transform":2089},"translate(7139,0)",[220,2091],{"dataC":237,"xLinkHref":2092},"#MJX-24-TEX-N-2212",[208,2094,2096],{"dataMmlNode":226,"transform":2095},"translate(8139.2,0)",[220,2097],{"dataC":876,"xLinkHref":2098},"#MJX-24-TEX-I-210E",[208,2100,2102],{"dataMmlNode":218,"transform":2101},"translate(8715.2,0)",[220,2103],{"dataC":222,"xLinkHref":2062},[208,2105,2107,2112],{"dataMmlNode":630,"transform":2106},"translate(9104.2,0)",[208,2108,2109],{"dataMmlNode":226},[220,2110],{"dataC":230,"xLinkHref":2111},"#MJX-24-TEX-I-1D465",[208,2113,2114,2118,2122],{"dataMmlNode":269,"transform":1387,"dataMjxTexclass":270},[208,2115,2116],{"dataMmlNode":218},[220,2117],{"dataC":222,"xLinkHref":2062},[208,2119,2120],{"dataMmlNode":226,"transform":227},[220,2121],{"dataC":251,"xLinkHref":2081},[208,2123,2124],{"dataMmlNode":218,"transform":651},[220,2125],{"dataC":265,"xLinkHref":2086},[208,2127,2129],{"dataMmlNode":218,"transform":2128},"translate(10553.3,0)",[220,2130],{"dataC":265,"xLinkHref":2086},[208,2132,2134],{"dataMmlNode":218,"transform":2133},"translate(10942.3,0)",[220,2135],{"dataC":265,"xLinkHref":2086},[208,2137,2140,2144,2159],{"dataMmlNode":2138,"transform":2139},"msubsup","translate(11331.3,0)",[208,2141,2142],{"dataMmlNode":226},[220,2143],{"dataC":230,"xLinkHref":2111},[208,2145,2147,2151,2155],{"dataMmlNode":269,"transform":2146,"dataMjxTexclass":270},"translate(605,530.4) scale(0.707)",[208,2148,2149],{"dataMmlNode":218},[220,2150],{"dataC":222,"xLinkHref":2062},[208,2152,2153],{"dataMmlNode":226,"transform":227},[220,2154],{"dataC":251,"xLinkHref":2081},[208,2156,2157],{"dataMmlNode":218,"transform":651},[220,2158],{"dataC":265,"xLinkHref":2086},[208,2160,2162],{"dataMmlNode":226,"transform":2161},"translate(605,-293.8) scale(0.707)",[220,2163],{"dataC":1775,"xLinkHref":2025},[16,2165,2166,2167,2170,2171,2174],{},"This is called the ",[20,2168,2169],{},"LMS (Least Mean Squares) update rule",", also known as the ",[20,2172,2173],{},"Widrow-Hoff learning rule",". Notice something elegant: the size of the update is proportional to the error. When the prediction is close to the actual value, the adjustment is tiny. When the error is large, the adjustment is large.",[69,2176,2178,2181],{"className":2177},[72,73],[75,2179,2180],{},"Deriving the equation",[79,2182,2184,2187,2318,2344,2544,2547,2813,2818,2821,3103,3108,3112,3378,3633,3832,4050,4186,4190,4193,4447,4451,4454,4694,4844,4849,5093,5099],{"className":2183},[82],[16,2185,2186],{},"The goal is to find the partial derivative for our update rule:",[151,2188,2190],{"className":2189,"jax":155,"display":1017},[154],[157,2191,2192,2224],{"style":1720,"xmlns":160,"width":1721,"height":1722,"role":163,"focusable":164,"viewBox":1723,"xmlnsXLink":166},[168,2193,2194,2197,2200,2203,2206,2209,2212,2215,2218,2221],{},[171,2195],{"id":2196,"d":1043},"MJX-25-TEX-I-1D703",[171,2198],{"id":2199,"d":1732},"MJX-25-TEX-I-1D457",[171,2201],{"id":2202,"d":1736},"MJX-25-TEX-N-3A",[171,2204],{"id":2205,"d":1039},"MJX-25-TEX-N-3D",[171,2207],{"id":2208,"d":182},"MJX-25-TEX-N-2212",[171,2210],{"id":2211,"d":1746},"MJX-25-TEX-I-1D6FC",[171,2213],{"id":2214,"d":1750},"MJX-25-TEX-I-1D715",[171,2216],{"id":2217,"d":1250},"MJX-25-TEX-I-1D43D",[171,2219],{"id":2220,"d":174},"MJX-25-TEX-N-28",[171,2222],{"id":2223,"d":198},"MJX-25-TEX-N-29",[208,2225,2226],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,2227,2228,2240,2252,2262,2267,2272],{"dataMmlNode":215},[208,2229,2230,2235],{"dataMmlNode":1093},[208,2231,2232],{"dataMmlNode":226},[220,2233],{"dataC":1099,"xLinkHref":2234},"#MJX-25-TEX-I-1D703",[208,2236,2237],{"dataMmlNode":226,"transform":1104},[220,2238],{"dataC":1775,"xLinkHref":2239},"#MJX-25-TEX-I-1D457",[208,2241,2242,2247],{"dataMmlNode":218,"transform":1779},[208,2243,2244],{"dataMmlNode":980},[220,2245],{"dataC":1784,"xLinkHref":2246},"#MJX-25-TEX-N-3A",[208,2248,2249],{"dataMmlNode":980,"transform":1788},[220,2250],{"dataC":1089,"xLinkHref":2251},"#MJX-25-TEX-N-3D",[208,2253,2254,2258],{"dataMmlNode":1093,"transform":1794},[208,2255,2256],{"dataMmlNode":226},[220,2257],{"dataC":1099,"xLinkHref":2234},[208,2259,2260],{"dataMmlNode":226,"transform":1104},[220,2261],{"dataC":1775,"xLinkHref":2239},[208,2263,2264],{"dataMmlNode":218,"transform":1805},[220,2265],{"dataC":237,"xLinkHref":2266},"#MJX-25-TEX-N-2212",[208,2268,2269],{"dataMmlNode":226,"transform":1811},[220,2270],{"dataC":1814,"xLinkHref":2271},"#MJX-25-TEX-I-1D6FC",[208,2273,2274,2300,2316],{"dataMmlNode":1324,"transform":1818},[208,2275,2276,2281,2286,2291,2295],{"dataMmlNode":1821,"transform":1822},[208,2277,2278],{"dataMmlNode":226},[220,2279],{"dataC":1827,"xLinkHref":2280},"#MJX-25-TEX-I-1D715",[208,2282,2283],{"dataMmlNode":226,"transform":1831},[220,2284],{"dataC":1296,"xLinkHref":2285},"#MJX-25-TEX-I-1D43D",[208,2287,2288],{"dataMmlNode":218,"transform":1837},[220,2289],{"dataC":222,"xLinkHref":2290},"#MJX-25-TEX-N-28",[208,2292,2293],{"dataMmlNode":226,"transform":1843},[220,2294],{"dataC":1099,"xLinkHref":2234},[208,2296,2297],{"dataMmlNode":218,"transform":1848},[220,2298],{"dataC":265,"xLinkHref":2299},"#MJX-25-TEX-N-29",[208,2301,2302,2306],{"dataMmlNode":1821,"transform":1854},[208,2303,2304],{"dataMmlNode":226},[220,2305],{"dataC":1827,"xLinkHref":2280},[208,2307,2308,2312],{"dataMmlNode":1093,"transform":1831},[208,2309,2310],{"dataMmlNode":226},[220,2311],{"dataC":1099,"xLinkHref":2234},[208,2313,2314],{"dataMmlNode":226,"transform":1104},[220,2315],{"dataC":1775,"xLinkHref":2239},[1339,2317],{"width":1871,"height":1342,"x":1343,"y":1344},[16,2319,2320,2321,2343],{},"We start with our Least Squares Cost Function (summing over all ",[151,2322,2324],{"className":2323,"jax":155},[154],[157,2325,2329,2334],{"style":606,"xmlns":160,"width":2326,"height":2327,"role":163,"focusable":164,"viewBox":2328,"xmlnsXLink":166},"0.781ex","1.52ex","0 -661 345 672",[168,2330,2331],{},[171,2332],{"id":2333,"d":190},"MJX-26-TEX-I-1D456",[208,2335,2336],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,2337,2338],{"dataMmlNode":215},[208,2339,2340],{"dataMmlNode":226},[220,2341],{"dataC":251,"xLinkHref":2342},"#MJX-26-TEX-I-1D456"," examples):",[151,2345,2347],{"className":2346,"jax":155,"display":1017},[154],[157,2348,2352,2401],{"style":1241,"xmlns":160,"width":2349,"height":2350,"role":163,"focusable":164,"viewBox":2351,"xmlnsXLink":166},"29.308ex","5.746ex","0 -1347.5 12954 2539.7",[168,2353,2354,2357,2360,2363,2366,2369,2372,2375,2378,2381,2385,2388,2391,2394,2397],{},[171,2355],{"id":2356,"d":1250},"MJX-27-TEX-I-1D43D",[171,2358],{"id":2359,"d":174},"MJX-27-TEX-N-28",[171,2361],{"id":2362,"d":1043},"MJX-27-TEX-I-1D703",[171,2364],{"id":2365,"d":198},"MJX-27-TEX-N-29",[171,2367],{"id":2368,"d":1039},"MJX-27-TEX-N-3D",[171,2370],{"id":2371,"d":1055},"MJX-27-TEX-N-31",[171,2373],{"id":2374,"d":1059},"MJX-27-TEX-N-32",[171,2376],{"id":2377,"d":1272},"MJX-27-TEX-LO-2211",[171,2379],{"id":2380,"d":190},"MJX-27-TEX-I-1D456",[171,2382],{"id":2383,"d":2384},"MJX-27-TEX-LO-28","M180 96T180 250T205 541T266 770T353 944T444 1069T527 1150H555Q561 1144 561 1141Q561 1137 545 1120T504 1072T447 995T386 878T330 721T288 513T272 251Q272 133 280 56Q293 -87 326 -209T399 -405T475 -531T536 -609T561 -640Q561 -643 555 -649H527Q483 -612 443 -568T353 -443T266 -270T205 -41Z",[171,2386],{"id":2387,"d":858},"MJX-27-TEX-I-210E",[171,2389],{"id":2390,"d":178},"MJX-27-TEX-I-1D465",[171,2392],{"id":2393,"d":182},"MJX-27-TEX-N-2212",[171,2395],{"id":2396,"d":674},"MJX-27-TEX-I-1D466",[171,2398],{"id":2399,"d":2400},"MJX-27-TEX-LO-29","M35 1138Q35 1150 51 1150H56H69Q113 1113 153 1069T243 944T330 771T391 541T416 250T391 -40T330 -270T243 -443T152 -568T69 -649H56Q43 -649 39 -647T35 -637Q65 -607 110 -548Q283 -316 316 56Q324 133 324 251Q324 368 316 445Q278 877 48 1123Q36 1137 35 1138Z",[208,2402,2403],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,2404,2405,2410,2415,2420,2425,2430,2444,2456],{"dataMmlNode":215},[208,2406,2407],{"dataMmlNode":226},[220,2408],{"dataC":1296,"xLinkHref":2409},"#MJX-27-TEX-I-1D43D",[208,2411,2412],{"dataMmlNode":218,"transform":1300},[220,2413],{"dataC":222,"xLinkHref":2414},"#MJX-27-TEX-N-28",[208,2416,2417],{"dataMmlNode":226,"transform":1306},[220,2418],{"dataC":1099,"xLinkHref":2419},"#MJX-27-TEX-I-1D703",[208,2421,2422],{"dataMmlNode":218,"transform":1312},[220,2423],{"dataC":265,"xLinkHref":2424},"#MJX-27-TEX-N-29",[208,2426,2427],{"dataMmlNode":218,"transform":1318},[220,2428],{"dataC":1089,"xLinkHref":2429},"#MJX-27-TEX-N-3D",[208,2431,2432,2437,2442],{"dataMmlNode":1324,"transform":1325},[208,2433,2434],{"dataMmlNode":1103,"transform":1328},[220,2435],{"dataC":1127,"xLinkHref":2436},"#MJX-27-TEX-N-31",[208,2438,2439],{"dataMmlNode":1103,"transform":1334},[220,2440],{"dataC":1157,"xLinkHref":2441},"#MJX-27-TEX-N-32",[1339,2443],{"width":1341,"height":1342,"x":1343,"y":1344},[208,2445,2446,2451],{"dataMmlNode":1347,"transform":1348},[208,2447,2448],{"dataMmlNode":218},[220,2449],{"dataC":1353,"xLinkHref":2450},"#MJX-27-TEX-LO-2211",[208,2452,2453],{"dataMmlNode":226,"transform":1357},[220,2454],{"dataC":251,"xLinkHref":2455},"#MJX-27-TEX-I-1D456",[208,2457,2459,2539],{"dataMmlNode":630,"transform":2458},"translate(5930.9,0)",[208,2460,2461,2467,2473,2478,2500,2505,2511,2533],{"dataMmlNode":1821},[208,2462,2464],{"dataMmlNode":218,"transform":2463},"translate(0 -0.5)",[220,2465],{"dataC":222,"xLinkHref":2466},"#MJX-27-TEX-LO-28",[208,2468,2470],{"dataMmlNode":226,"transform":2469},"translate(597,0)",[220,2471],{"dataC":876,"xLinkHref":2472},"#MJX-27-TEX-I-210E",[208,2474,2476],{"dataMmlNode":218,"transform":2475},"translate(1173,0)",[220,2477],{"dataC":222,"xLinkHref":2414},[208,2479,2481,2486],{"dataMmlNode":630,"transform":2480},"translate(1562,0)",[208,2482,2483],{"dataMmlNode":226},[220,2484],{"dataC":230,"xLinkHref":2485},"#MJX-27-TEX-I-1D465",[208,2487,2488,2492,2496],{"dataMmlNode":269,"transform":1387,"dataMjxTexclass":270},[208,2489,2490],{"dataMmlNode":218},[220,2491],{"dataC":222,"xLinkHref":2414},[208,2493,2494],{"dataMmlNode":226,"transform":227},[220,2495],{"dataC":251,"xLinkHref":2455},[208,2497,2498],{"dataMmlNode":218,"transform":651},[220,2499],{"dataC":265,"xLinkHref":2424},[208,2501,2503],{"dataMmlNode":218,"transform":2502},"translate(3011.1,0)",[220,2504],{"dataC":265,"xLinkHref":2424},[208,2506,2508],{"dataMmlNode":218,"transform":2507},"translate(3622.3,0)",[220,2509],{"dataC":237,"xLinkHref":2510},"#MJX-27-TEX-N-2212",[208,2512,2514,2519],{"dataMmlNode":630,"transform":2513},"translate(4622.5,0)",[208,2515,2516],{"dataMmlNode":226},[220,2517],{"dataC":694,"xLinkHref":2518},"#MJX-27-TEX-I-1D466",[208,2520,2521,2525,2529],{"dataMmlNode":269,"transform":1421,"dataMjxTexclass":270},[208,2522,2523],{"dataMmlNode":218},[220,2524],{"dataC":222,"xLinkHref":2414},[208,2526,2527],{"dataMmlNode":226,"transform":227},[220,2528],{"dataC":251,"xLinkHref":2455},[208,2530,2531],{"dataMmlNode":218,"transform":651},[220,2532],{"dataC":265,"xLinkHref":2424},[208,2534,2536],{"dataMmlNode":218,"transform":2535},"translate(5989.6,0) translate(0 -0.5)",[220,2537],{"dataC":265,"xLinkHref":2538},"#MJX-27-TEX-LO-29",[208,2540,2542],{"dataMmlNode":1103,"transform":2541},"translate(6619.6,876.6) scale(0.707)",[220,2543],{"dataC":1157,"xLinkHref":2441},[16,2545,2546],{},"And our linear hypothesis:",[151,2548,2550],{"className":2549,"jax":155,"display":1017},[154],[157,2551,2554,2600],{"style":1965,"xmlns":160,"width":2552,"height":1967,"role":163,"focusable":164,"viewBox":2553,"xmlnsXLink":166},"46.38ex","0 -1060.7 20500.1 1498.8",[168,2555,2556,2559,2562,2565,2568,2571,2574,2577,2580,2583,2586,2589,2593,2596],{},[171,2557],{"id":2558,"d":858},"MJX-28-TEX-I-210E",[171,2560],{"id":2561,"d":174},"MJX-28-TEX-N-28",[171,2563],{"id":2564,"d":178},"MJX-28-TEX-I-1D465",[171,2566],{"id":2567,"d":190},"MJX-28-TEX-I-1D456",[171,2569],{"id":2570,"d":198},"MJX-28-TEX-N-29",[171,2572],{"id":2573,"d":1039},"MJX-28-TEX-N-3D",[171,2575],{"id":2576,"d":1043},"MJX-28-TEX-I-1D703",[171,2578],{"id":2579,"d":1047},"MJX-28-TEX-N-30",[171,2581],{"id":2582,"d":1051},"MJX-28-TEX-N-2B",[171,2584],{"id":2585,"d":1055},"MJX-28-TEX-N-31",[171,2587],{"id":2588,"d":1059},"MJX-28-TEX-N-32",[171,2590],{"id":2591,"d":2592},"MJX-28-TEX-N-22EF","M78 250Q78 274 95 292T138 310Q162 310 180 294T199 251Q199 226 182 208T139 190T96 207T78 250ZM525 250Q525 274 542 292T585 310Q609 310 627 294T646 251Q646 226 629 208T586 190T543 207T525 250ZM972 250Q972 274 989 292T1032 310Q1056 310 1074 294T1093 251Q1093 226 1076 208T1033 190T990 207T972 250Z",[171,2594],{"id":2595,"d":1732},"MJX-28-TEX-I-1D457",[171,2597],{"id":2598,"d":2599},"MJX-28-TEX-N-2026","M78 60Q78 84 95 102T138 120Q162 120 180 104T199 61Q199 36 182 18T139 0T96 17T78 60ZM525 60Q525 84 542 102T585 120Q609 120 627 104T646 61Q646 36 629 18T586 0T543 17T525 60ZM972 60Q972 84 989 102T1032 120Q1056 120 1074 104T1093 61Q1093 36 1076 18T1033 0T990 17T972 60Z",[208,2601,2602],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,2603,2604,2609,2614,2637,2642,2648,2661,2667,2679,2705,2710,2722,2747,2752,2759,2764,2776,2801,2806],{"dataMmlNode":215},[208,2605,2606],{"dataMmlNode":226},[220,2607],{"dataC":876,"xLinkHref":2608},"#MJX-28-TEX-I-210E",[208,2610,2611],{"dataMmlNode":218,"transform":880},[220,2612],{"dataC":222,"xLinkHref":2613},"#MJX-28-TEX-N-28",[208,2615,2616,2621],{"dataMmlNode":630,"transform":886},[208,2617,2618],{"dataMmlNode":226},[220,2619],{"dataC":230,"xLinkHref":2620},"#MJX-28-TEX-I-1D465",[208,2622,2623,2627,2632],{"dataMmlNode":269,"transform":1387,"dataMjxTexclass":270},[208,2624,2625],{"dataMmlNode":218},[220,2626],{"dataC":222,"xLinkHref":2613},[208,2628,2629],{"dataMmlNode":226,"transform":227},[220,2630],{"dataC":251,"xLinkHref":2631},"#MJX-28-TEX-I-1D456",[208,2633,2634],{"dataMmlNode":218,"transform":651},[220,2635],{"dataC":265,"xLinkHref":2636},"#MJX-28-TEX-N-29",[208,2638,2640],{"dataMmlNode":218,"transform":2639},"translate(2414.1,0)",[220,2641],{"dataC":265,"xLinkHref":2636},[208,2643,2645],{"dataMmlNode":218,"transform":2644},"translate(3080.9,0)",[220,2646],{"dataC":1089,"xLinkHref":2647},"#MJX-28-TEX-N-3D",[208,2649,2651,2656],{"dataMmlNode":1093,"transform":2650},"translate(4136.6,0)",[208,2652,2653],{"dataMmlNode":226},[220,2654],{"dataC":1099,"xLinkHref":2655},"#MJX-28-TEX-I-1D703",[208,2657,2658],{"dataMmlNode":1103,"transform":1104},[220,2659],{"dataC":1107,"xLinkHref":2660},"#MJX-28-TEX-N-30",[208,2662,2664],{"dataMmlNode":218,"transform":2663},"translate(5264.4,0)",[220,2665],{"dataC":1114,"xLinkHref":2666},"#MJX-28-TEX-N-2B",[208,2668,2670,2674],{"dataMmlNode":1093,"transform":2669},"translate(6264.6,0)",[208,2671,2672],{"dataMmlNode":226},[220,2673],{"dataC":1099,"xLinkHref":2655},[208,2675,2676],{"dataMmlNode":1103,"transform":1104},[220,2677],{"dataC":1127,"xLinkHref":2678},"#MJX-28-TEX-N-31",[208,2680,2682,2686,2700],{"dataMmlNode":2138,"transform":2681},"translate(7170.2,0)",[208,2683,2684],{"dataMmlNode":226},[220,2685],{"dataC":230,"xLinkHref":2620},[208,2687,2688,2692,2696],{"dataMmlNode":269,"transform":2146,"dataMjxTexclass":270},[208,2689,2690],{"dataMmlNode":218},[220,2691],{"dataC":222,"xLinkHref":2613},[208,2693,2694],{"dataMmlNode":226,"transform":227},[220,2695],{"dataC":251,"xLinkHref":2631},[208,2697,2698],{"dataMmlNode":218,"transform":651},[220,2699],{"dataC":265,"xLinkHref":2636},[208,2701,2703],{"dataMmlNode":1103,"transform":2702},"translate(605,-297.3) scale(0.707)",[220,2704],{"dataC":1127,"xLinkHref":2678},[208,2706,2708],{"dataMmlNode":218,"transform":2707},"translate(8841.5,0)",[220,2709],{"dataC":1114,"xLinkHref":2666},[208,2711,2713,2717],{"dataMmlNode":1093,"transform":2712},"translate(9841.7,0)",[208,2714,2715],{"dataMmlNode":226},[220,2716],{"dataC":1099,"xLinkHref":2655},[208,2718,2719],{"dataMmlNode":1103,"transform":1104},[220,2720],{"dataC":1157,"xLinkHref":2721},"#MJX-28-TEX-N-32",[208,2723,2725,2729,2743],{"dataMmlNode":2138,"transform":2724},"translate(10747.3,0)",[208,2726,2727],{"dataMmlNode":226},[220,2728],{"dataC":230,"xLinkHref":2620},[208,2730,2731,2735,2739],{"dataMmlNode":269,"transform":2146,"dataMjxTexclass":270},[208,2732,2733],{"dataMmlNode":218},[220,2734],{"dataC":222,"xLinkHref":2613},[208,2736,2737],{"dataMmlNode":226,"transform":227},[220,2738],{"dataC":251,"xLinkHref":2631},[208,2740,2741],{"dataMmlNode":218,"transform":651},[220,2742],{"dataC":265,"xLinkHref":2636},[208,2744,2745],{"dataMmlNode":1103,"transform":2702},[220,2746],{"dataC":1157,"xLinkHref":2721},[208,2748,2750],{"dataMmlNode":218,"transform":2749},"translate(12418.6,0)",[220,2751],{"dataC":1114,"xLinkHref":2666},[208,2753,2755],{"dataMmlNode":218,"transform":2754},"translate(13418.8,0)",[220,2756],{"dataC":2757,"xLinkHref":2758},"22EF","#MJX-28-TEX-N-22EF",[208,2760,2762],{"dataMmlNode":218,"transform":2761},"translate(14813,0)",[220,2763],{"dataC":1114,"xLinkHref":2666},[208,2765,2767,2771],{"dataMmlNode":1093,"transform":2766},"translate(15813.2,0)",[208,2768,2769],{"dataMmlNode":226},[220,2770],{"dataC":1099,"xLinkHref":2655},[208,2772,2773],{"dataMmlNode":226,"transform":1104},[220,2774],{"dataC":1775,"xLinkHref":2775},"#MJX-28-TEX-I-1D457",[208,2777,2779,2783,2797],{"dataMmlNode":2138,"transform":2778},"translate(16656.6,0)",[208,2780,2781],{"dataMmlNode":226},[220,2782],{"dataC":230,"xLinkHref":2620},[208,2784,2785,2789,2793],{"dataMmlNode":269,"transform":2146,"dataMjxTexclass":270},[208,2786,2787],{"dataMmlNode":218},[220,2788],{"dataC":222,"xLinkHref":2613},[208,2790,2791],{"dataMmlNode":226,"transform":227},[220,2792],{"dataC":251,"xLinkHref":2631},[208,2794,2795],{"dataMmlNode":218,"transform":651},[220,2796],{"dataC":265,"xLinkHref":2636},[208,2798,2799],{"dataMmlNode":226,"transform":2161},[220,2800],{"dataC":1775,"xLinkHref":2775},[208,2802,2804],{"dataMmlNode":218,"transform":2803},"translate(18327.9,0)",[220,2805],{"dataC":1114,"xLinkHref":2666},[208,2807,2809],{"dataMmlNode":218,"transform":2808},"translate(19328.1,0)",[220,2810],{"dataC":2811,"xLinkHref":2812},"2026","#MJX-28-TEX-N-2026",[2814,2815,2817],"h4",{"id":2816},"set-up-the-explicit-chain-rule","Set up the Explicit Chain Rule",[16,2819,2820],{},"Instead of differentiating everything at once, we split the derivative into two clear parts using Leibniz notation:",[151,2822,2824],{"className":2823,"jax":155,"display":1017},[154],[157,2825,2830,2877],{"style":2826,"xmlns":160,"width":2827,"height":2828,"role":163,"focusable":164,"viewBox":2829,"xmlnsXLink":166},"vertical-align: -2.827ex;","35.644ex","6.785ex","0 -1749.5 15754.8 2999",[168,2831,2832,2835,2838,2841,2844,2847,2850,2853,2856,2859,2863,2866,2869,2873],{},[171,2833],{"id":2834,"d":1750},"MJX-29-TEX-I-1D715",[171,2836],{"id":2837,"d":1250},"MJX-29-TEX-I-1D43D",[171,2839],{"id":2840,"d":174},"MJX-29-TEX-N-28",[171,2842],{"id":2843,"d":1043},"MJX-29-TEX-I-1D703",[171,2845],{"id":2846,"d":198},"MJX-29-TEX-N-29",[171,2848],{"id":2849,"d":1732},"MJX-29-TEX-I-1D457",[171,2851],{"id":2852,"d":1039},"MJX-29-TEX-N-3D",[171,2854],{"id":2855,"d":1272},"MJX-29-TEX-LO-2211",[171,2857],{"id":2858,"d":190},"MJX-29-TEX-I-1D456",[171,2860],{"id":2861,"d":2862},"MJX-29-TEX-S4-28","M758 -1237T758 -1240T752 -1249H736Q718 -1249 717 -1248Q711 -1245 672 -1199Q237 -706 237 251T672 1700Q697 1730 716 1749Q718 1750 735 1750H752Q758 1744 758 1741Q758 1737 740 1713T689 1644T619 1537T540 1380T463 1176Q348 802 348 251Q348 -242 441 -599T744 -1218Q758 -1237 758 -1240Z",[171,2864],{"id":2865,"d":858},"MJX-29-TEX-I-210E",[171,2867],{"id":2868,"d":178},"MJX-29-TEX-I-1D465",[171,2870],{"id":2871,"d":2872},"MJX-29-TEX-N-22C5","M78 250Q78 274 95 292T138 310Q162 310 180 294T199 251Q199 226 182 208T139 190T96 207T78 250Z",[171,2874],{"id":2875,"d":2876},"MJX-29-TEX-S4-29","M33 1741Q33 1750 51 1750H60H65Q73 1750 81 1743T119 1700Q554 1207 554 251Q554 -707 119 -1199Q76 -1250 66 -1250Q65 -1250 62 -1250T56 -1249Q55 -1249 53 -1249T49 -1250Q33 -1250 33 -1239Q33 -1236 50 -1214T98 -1150T163 -1052T238 -910T311 -727Q443 -335 443 251Q443 402 436 532T405 831T339 1142T224 1438T50 1716Q33 1737 33 1741Z",[208,2878,2879],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,2880,2881,2929,2935,2948],{"dataMmlNode":215},[208,2882,2883,2910,2927],{"dataMmlNode":1324},[208,2884,2885,2890,2895,2900,2905],{"dataMmlNode":1821,"transform":1822},[208,2886,2887],{"dataMmlNode":226},[220,2888],{"dataC":1827,"xLinkHref":2889},"#MJX-29-TEX-I-1D715",[208,2891,2892],{"dataMmlNode":226,"transform":1831},[220,2893],{"dataC":1296,"xLinkHref":2894},"#MJX-29-TEX-I-1D43D",[208,2896,2897],{"dataMmlNode":218,"transform":1837},[220,2898],{"dataC":222,"xLinkHref":2899},"#MJX-29-TEX-N-28",[208,2901,2902],{"dataMmlNode":226,"transform":1843},[220,2903],{"dataC":1099,"xLinkHref":2904},"#MJX-29-TEX-I-1D703",[208,2906,2907],{"dataMmlNode":218,"transform":1848},[220,2908],{"dataC":265,"xLinkHref":2909},"#MJX-29-TEX-N-29",[208,2911,2912,2916],{"dataMmlNode":1821,"transform":1854},[208,2913,2914],{"dataMmlNode":226},[220,2915],{"dataC":1827,"xLinkHref":2889},[208,2917,2918,2922],{"dataMmlNode":1093,"transform":1831},[208,2919,2920],{"dataMmlNode":226},[220,2921],{"dataC":1099,"xLinkHref":2904},[208,2923,2924],{"dataMmlNode":226,"transform":1104},[220,2925],{"dataC":1775,"xLinkHref":2926},"#MJX-29-TEX-I-1D457",[1339,2928],{"width":1871,"height":1342,"x":1343,"y":1344},[208,2930,2932],{"dataMmlNode":218,"transform":2931},"translate(3163.8,0)",[220,2933],{"dataC":1089,"xLinkHref":2934},"#MJX-29-TEX-N-3D",[208,2936,2938,2943],{"dataMmlNode":1347,"transform":2937},"translate(4219.6,0)",[208,2939,2940],{"dataMmlNode":218},[220,2941],{"dataC":1353,"xLinkHref":2942},"#MJX-29-TEX-LO-2211",[208,2944,2945],{"dataMmlNode":226,"transform":1357},[220,2946],{"dataC":251,"xLinkHref":2947},"#MJX-29-TEX-I-1D456",[208,2949,2951,2956,3030,3037,3097],{"dataMmlNode":1821,"transform":2950},"translate(5830.2,0)",[208,2952,2953],{"dataMmlNode":218,"transform":2463},[220,2954],{"dataC":222,"xLinkHref":2955},"#MJX-29-TEX-S4-28",[208,2957,2959,2982,3027],{"dataMmlNode":1324,"transform":2958},"translate(792,0)",[208,2960,2962,2966,2970,2974,2978],{"dataMmlNode":1821,"transform":2961},"translate(681.5,710)",[208,2963,2964],{"dataMmlNode":226},[220,2965],{"dataC":1827,"xLinkHref":2889},[208,2967,2968],{"dataMmlNode":226,"transform":1831},[220,2969],{"dataC":1296,"xLinkHref":2894},[208,2971,2972],{"dataMmlNode":218,"transform":1837},[220,2973],{"dataC":222,"xLinkHref":2899},[208,2975,2976],{"dataMmlNode":226,"transform":1843},[220,2977],{"dataC":1099,"xLinkHref":2904},[208,2979,2980],{"dataMmlNode":218,"transform":1848},[220,2981],{"dataC":265,"xLinkHref":2909},[208,2983,2985,2989,2994,2999,3022],{"dataMmlNode":1821,"transform":2984},"translate(220,-779.3)",[208,2986,2987],{"dataMmlNode":226},[220,2988],{"dataC":1827,"xLinkHref":2889},[208,2990,2991],{"dataMmlNode":226,"transform":1831},[220,2992],{"dataC":876,"xLinkHref":2993},"#MJX-29-TEX-I-210E",[208,2995,2997],{"dataMmlNode":218,"transform":2996},"translate(1142,0)",[220,2998],{"dataC":222,"xLinkHref":2899},[208,3000,3002,3007],{"dataMmlNode":630,"transform":3001},"translate(1531,0)",[208,3003,3004],{"dataMmlNode":226},[220,3005],{"dataC":230,"xLinkHref":3006},"#MJX-29-TEX-I-1D465",[208,3008,3010,3014,3018],{"dataMmlNode":269,"transform":3009,"dataMjxTexclass":270},"translate(605,289) scale(0.707)",[208,3011,3012],{"dataMmlNode":218},[220,3013],{"dataC":222,"xLinkHref":2899},[208,3015,3016],{"dataMmlNode":226,"transform":227},[220,3017],{"dataC":251,"xLinkHref":2947},[208,3019,3020],{"dataMmlNode":218,"transform":651},[220,3021],{"dataC":265,"xLinkHref":2909},[208,3023,3025],{"dataMmlNode":218,"transform":3024},"translate(2980.1,0)",[220,3026],{"dataC":265,"xLinkHref":2909},[1339,3028],{"width":3029,"height":1342,"x":1343,"y":1344},3569.1,[208,3031,3033],{"dataMmlNode":218,"transform":3032},"translate(4823.3,0)",[220,3034],{"dataC":3035,"xLinkHref":3036},"22C5","#MJX-29-TEX-N-22C5",[208,3038,3040,3078,3095],{"dataMmlNode":1324,"transform":3039},"translate(5323.5,0)",[208,3041,3042,3046,3050,3054,3074],{"dataMmlNode":1821,"transform":1822},[208,3043,3044],{"dataMmlNode":226},[220,3045],{"dataC":1827,"xLinkHref":2889},[208,3047,3048],{"dataMmlNode":226,"transform":1831},[220,3049],{"dataC":876,"xLinkHref":2993},[208,3051,3052],{"dataMmlNode":218,"transform":2996},[220,3053],{"dataC":222,"xLinkHref":2899},[208,3055,3056,3060],{"dataMmlNode":630,"transform":3001},[208,3057,3058],{"dataMmlNode":226},[220,3059],{"dataC":230,"xLinkHref":3006},[208,3061,3062,3066,3070],{"dataMmlNode":269,"transform":638,"dataMjxTexclass":270},[208,3063,3064],{"dataMmlNode":218},[220,3065],{"dataC":222,"xLinkHref":2899},[208,3067,3068],{"dataMmlNode":226,"transform":227},[220,3069],{"dataC":251,"xLinkHref":2947},[208,3071,3072],{"dataMmlNode":218,"transform":651},[220,3073],{"dataC":265,"xLinkHref":2909},[208,3075,3076],{"dataMmlNode":218,"transform":3024},[220,3077],{"dataC":265,"xLinkHref":2909},[208,3079,3081,3085],{"dataMmlNode":1821,"transform":3080},"translate(1199.9,-686)",[208,3082,3083],{"dataMmlNode":226},[220,3084],{"dataC":1827,"xLinkHref":2889},[208,3086,3087,3091],{"dataMmlNode":1093,"transform":1831},[208,3088,3089],{"dataMmlNode":226},[220,3090],{"dataC":1099,"xLinkHref":2904},[208,3092,3093],{"dataMmlNode":226,"transform":1104},[220,3094],{"dataC":1775,"xLinkHref":2926},[1339,3096],{"width":3029,"height":1342,"x":1343,"y":1344},[208,3098,3100],{"dataMmlNode":218,"transform":3099},"translate(9132.6,0) translate(0 -0.5)",[220,3101],{"dataC":265,"xLinkHref":3102},"#MJX-29-TEX-S4-29",[16,3104,3105],{},[87,3106,3107],{},"We put the derivatives inside the sum because the total gradient is just the sum of the gradients of each individual training example.",[2814,3109,3111],{"id":3110},"calculate-the-two-parts","Calculate the Two Parts",[16,3113,3114,3117,3118,3240,3241,3298,3299,3330,3331,3354,3355,3377],{},[20,3115,3116],{},"Part 1: How does the cost change with respect to the prediction?","\nLet ",[151,3119,3121],{"className":3120,"jax":155},[154],[157,3122,3125,3155],{"style":159,"xmlns":160,"width":3123,"height":725,"role":163,"focusable":164,"viewBox":3124,"xmlnsXLink":166},"16.512ex","0 -893.3 7298.2 1143.3",[168,3126,3127,3131,3134,3137,3140,3143,3146,3149,3152],{},[171,3128],{"id":3129,"d":3130},"MJX-30-TEX-I-1D462","M21 287Q21 295 30 318T55 370T99 420T158 442Q204 442 227 417T250 358Q250 340 216 246T182 105Q182 62 196 45T238 27T291 44T328 78L339 95Q341 99 377 247Q407 367 413 387T427 416Q444 431 463 431Q480 431 488 421T496 402L420 84Q419 79 419 68Q419 43 426 35T447 26Q469 29 482 57T512 145Q514 153 532 153Q551 153 551 144Q550 139 549 130T540 98T523 55T498 17T462 -8Q454 -10 438 -10Q372 -10 347 46Q345 45 336 36T318 21T296 6T267 -6T233 -11Q189 -11 155 7Q103 38 103 113Q103 170 138 262T173 379Q173 380 173 381Q173 390 173 393T169 400T158 404H154Q131 404 112 385T82 344T65 302T57 280Q55 278 41 278H27Q21 284 21 287Z",[171,3132],{"id":3133,"d":1039},"MJX-30-TEX-N-3D",[171,3135],{"id":3136,"d":858},"MJX-30-TEX-I-210E",[171,3138],{"id":3139,"d":174},"MJX-30-TEX-N-28",[171,3141],{"id":3142,"d":178},"MJX-30-TEX-I-1D465",[171,3144],{"id":3145,"d":190},"MJX-30-TEX-I-1D456",[171,3147],{"id":3148,"d":198},"MJX-30-TEX-N-29",[171,3150],{"id":3151,"d":182},"MJX-30-TEX-N-2212",[171,3153],{"id":3154,"d":674},"MJX-30-TEX-I-1D466",[208,3156,3157],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,3158,3159,3165,3171,3177,3183,3207,3212,3218],{"dataMmlNode":215},[208,3160,3161],{"dataMmlNode":226},[220,3162],{"dataC":3163,"xLinkHref":3164},"1D462","#MJX-30-TEX-I-1D462",[208,3166,3168],{"dataMmlNode":218,"transform":3167},"translate(849.8,0)",[220,3169],{"dataC":1089,"xLinkHref":3170},"#MJX-30-TEX-N-3D",[208,3172,3174],{"dataMmlNode":226,"transform":3173},"translate(1905.6,0)",[220,3175],{"dataC":876,"xLinkHref":3176},"#MJX-30-TEX-I-210E",[208,3178,3180],{"dataMmlNode":218,"transform":3179},"translate(2481.6,0)",[220,3181],{"dataC":222,"xLinkHref":3182},"#MJX-30-TEX-N-28",[208,3184,3186,3191],{"dataMmlNode":630,"transform":3185},"translate(2870.6,0)",[208,3187,3188],{"dataMmlNode":226},[220,3189],{"dataC":230,"xLinkHref":3190},"#MJX-30-TEX-I-1D465",[208,3192,3193,3197,3202],{"dataMmlNode":269,"transform":638,"dataMjxTexclass":270},[208,3194,3195],{"dataMmlNode":218},[220,3196],{"dataC":222,"xLinkHref":3182},[208,3198,3199],{"dataMmlNode":226,"transform":227},[220,3200],{"dataC":251,"xLinkHref":3201},"#MJX-30-TEX-I-1D456",[208,3203,3204],{"dataMmlNode":218,"transform":651},[220,3205],{"dataC":265,"xLinkHref":3206},"#MJX-30-TEX-N-29",[208,3208,3210],{"dataMmlNode":218,"transform":3209},"translate(4319.6,0)",[220,3211],{"dataC":265,"xLinkHref":3206},[208,3213,3215],{"dataMmlNode":218,"transform":3214},"translate(4930.9,0)",[220,3216],{"dataC":237,"xLinkHref":3217},"#MJX-30-TEX-N-2212",[208,3219,3221,3226],{"dataMmlNode":630,"transform":3220},"translate(5931.1,0)",[208,3222,3223],{"dataMmlNode":226},[220,3224],{"dataC":694,"xLinkHref":3225},"#MJX-30-TEX-I-1D466",[208,3227,3228,3232,3236],{"dataMmlNode":269,"transform":698,"dataMjxTexclass":270},[208,3229,3230],{"dataMmlNode":218},[220,3231],{"dataC":222,"xLinkHref":3182},[208,3233,3234],{"dataMmlNode":226,"transform":227},[220,3235],{"dataC":251,"xLinkHref":3201},[208,3237,3238],{"dataMmlNode":218,"transform":651},[220,3239],{"dataC":265,"xLinkHref":3206},". Using the power rule (",[151,3242,3244],{"className":3243,"jax":155},[154],[157,3245,3249,3264],{"style":606,"xmlns":160,"width":3246,"height":3247,"role":163,"focusable":164,"viewBox":3248,"xmlnsXLink":166},"5.982ex","1.742ex","0 -759 2643.9 770",[168,3250,3251,3254,3257,3260],{},[171,3252],{"id":3253,"d":1059},"MJX-31-TEX-N-32",[171,3255],{"id":3256,"d":3130},"MJX-31-TEX-I-1D462",[171,3258],{"id":3259,"d":2872},"MJX-31-TEX-N-22C5",[171,3261],{"id":3262,"d":3263},"MJX-31-TEX-V-2032","M79 43Q73 43 52 49T30 61Q30 68 85 293T146 528Q161 560 198 560Q218 560 240 545T262 501Q262 496 260 486Q259 479 173 263T84 45T79 43Z",[208,3265,3266],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,3267,3268,3273,3279,3285],{"dataMmlNode":215},[208,3269,3270],{"dataMmlNode":1103},[220,3271],{"dataC":1157,"xLinkHref":3272},"#MJX-31-TEX-N-32",[208,3274,3276],{"dataMmlNode":226,"transform":3275},"translate(500,0)",[220,3277],{"dataC":3163,"xLinkHref":3278},"#MJX-31-TEX-I-1D462",[208,3280,3282],{"dataMmlNode":218,"transform":3281},"translate(1294.2,0)",[220,3283],{"dataC":3035,"xLinkHref":3284},"#MJX-31-TEX-N-22C5",[208,3286,3288,3292],{"dataMmlNode":630,"transform":3287},"translate(1794.4,0)",[208,3289,3290],{"dataMmlNode":226},[220,3291],{"dataC":3163,"xLinkHref":3278},[208,3293,3294],{"dataMmlNode":218,"transform":638},[220,3295],{"dataC":3296,"xLinkHref":3297},"2032","#MJX-31-TEX-V-2032"," where ",[151,3300,3302],{"className":3301,"jax":155},[154],[157,3303,3306,3314],{"style":606,"xmlns":160,"width":3304,"height":3247,"role":163,"focusable":164,"viewBox":3305,"xmlnsXLink":166},"1.922ex","0 -759 849.5 770",[168,3307,3308,3311],{},[171,3309],{"id":3310,"d":3130},"MJX-32-TEX-I-1D462",[171,3312],{"id":3313,"d":3263},"MJX-32-TEX-V-2032",[208,3315,3316],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,3317,3318],{"dataMmlNode":215},[208,3319,3320,3325],{"dataMmlNode":630},[208,3321,3322],{"dataMmlNode":226},[220,3323],{"dataC":3163,"xLinkHref":3324},"#MJX-32-TEX-I-1D462",[208,3326,3327],{"dataMmlNode":218,"transform":638},[220,3328],{"dataC":3296,"xLinkHref":3329},"#MJX-32-TEX-V-2032"," is just ",[151,3332,3334],{"className":3333,"jax":155},[154],[157,3335,3340,3345],{"style":3336,"xmlns":160,"width":3337,"height":3338,"role":163,"focusable":164,"viewBox":3339,"xmlnsXLink":166},"vertical-align: 0;","1.131ex","1.507ex","0 -666 500 666",[168,3341,3342],{},[171,3343],{"id":3344,"d":1055},"MJX-33-TEX-N-31",[208,3346,3347],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,3348,3349],{"dataMmlNode":215},[208,3350,3351],{"dataMmlNode":1103},[220,3352],{"dataC":1127,"xLinkHref":3353},"#MJX-33-TEX-N-31"," with respect to ",[151,3356,3358],{"className":3357,"jax":155},[154],[157,3359,3363,3368],{"style":606,"xmlns":160,"width":3360,"height":3361,"role":163,"focusable":164,"viewBox":3362,"xmlnsXLink":166},"1.303ex","1.595ex","0 -694 576 705",[168,3364,3365],{},[171,3366],{"id":3367,"d":858},"MJX-34-TEX-I-210E",[208,3369,3370],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,3371,3372],{"dataMmlNode":215},[208,3373,3374],{"dataMmlNode":226},[220,3375],{"dataC":876,"xLinkHref":3376},"#MJX-34-TEX-I-210E","):",[151,3379,3381],{"className":3380,"jax":155,"display":1017},[154],[157,3382,3387,3437],{"style":3383,"xmlns":160,"width":3384,"height":3385,"role":163,"focusable":164,"viewBox":3386,"xmlnsXLink":166},"vertical-align: -2.329ex;","34.461ex","5.632ex","0 -1460 15231.9 2489.3",[168,3388,3389,3392,3395,3398,3401,3404,3407,3410,3413,3416,3419,3422,3425,3428,3431,3434],{},[171,3390],{"id":3391,"d":1750},"MJX-35-TEX-I-1D715",[171,3393],{"id":3394,"d":1250},"MJX-35-TEX-I-1D43D",[171,3396],{"id":3397,"d":174},"MJX-35-TEX-N-28",[171,3399],{"id":3400,"d":1043},"MJX-35-TEX-I-1D703",[171,3402],{"id":3403,"d":198},"MJX-35-TEX-N-29",[171,3405],{"id":3406,"d":858},"MJX-35-TEX-I-210E",[171,3408],{"id":3409,"d":178},"MJX-35-TEX-I-1D465",[171,3411],{"id":3412,"d":190},"MJX-35-TEX-I-1D456",[171,3414],{"id":3415,"d":1039},"MJX-35-TEX-N-3D",[171,3417],{"id":3418,"d":1055},"MJX-35-TEX-N-31",[171,3420],{"id":3421,"d":1059},"MJX-35-TEX-N-32",[171,3423],{"id":3424,"d":2872},"MJX-35-TEX-N-22C5",[171,3426],{"id":3427,"d":2384},"MJX-35-TEX-LO-28",[171,3429],{"id":3430,"d":182},"MJX-35-TEX-N-2212",[171,3432],{"id":3433,"d":674},"MJX-35-TEX-I-1D466",[171,3435],{"id":3436,"d":2400},"MJX-35-TEX-LO-29",[208,3438,3439],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,3440,3441,3513,3519,3534,3540,3545],{"dataMmlNode":215},[208,3442,3443,3470,3511],{"dataMmlNode":1324},[208,3444,3445,3450,3455,3460,3465],{"dataMmlNode":1821,"transform":2961},[208,3446,3447],{"dataMmlNode":226},[220,3448],{"dataC":1827,"xLinkHref":3449},"#MJX-35-TEX-I-1D715",[208,3451,3452],{"dataMmlNode":226,"transform":1831},[220,3453],{"dataC":1296,"xLinkHref":3454},"#MJX-35-TEX-I-1D43D",[208,3456,3457],{"dataMmlNode":218,"transform":1837},[220,3458],{"dataC":222,"xLinkHref":3459},"#MJX-35-TEX-N-28",[208,3461,3462],{"dataMmlNode":226,"transform":1843},[220,3463],{"dataC":1099,"xLinkHref":3464},"#MJX-35-TEX-I-1D703",[208,3466,3467],{"dataMmlNode":218,"transform":1848},[220,3468],{"dataC":265,"xLinkHref":3469},"#MJX-35-TEX-N-29",[208,3471,3472,3476,3481,3485,3507],{"dataMmlNode":1821,"transform":2984},[208,3473,3474],{"dataMmlNode":226},[220,3475],{"dataC":1827,"xLinkHref":3449},[208,3477,3478],{"dataMmlNode":226,"transform":1831},[220,3479],{"dataC":876,"xLinkHref":3480},"#MJX-35-TEX-I-210E",[208,3482,3483],{"dataMmlNode":218,"transform":2996},[220,3484],{"dataC":222,"xLinkHref":3459},[208,3486,3487,3492],{"dataMmlNode":630,"transform":3001},[208,3488,3489],{"dataMmlNode":226},[220,3490],{"dataC":230,"xLinkHref":3491},"#MJX-35-TEX-I-1D465",[208,3493,3494,3498,3503],{"dataMmlNode":269,"transform":3009,"dataMjxTexclass":270},[208,3495,3496],{"dataMmlNode":218},[220,3497],{"dataC":222,"xLinkHref":3459},[208,3499,3500],{"dataMmlNode":226,"transform":227},[220,3501],{"dataC":251,"xLinkHref":3502},"#MJX-35-TEX-I-1D456",[208,3504,3505],{"dataMmlNode":218,"transform":651},[220,3506],{"dataC":265,"xLinkHref":3469},[208,3508,3509],{"dataMmlNode":218,"transform":3024},[220,3510],{"dataC":265,"xLinkHref":3469},[1339,3512],{"width":3029,"height":1342,"x":1343,"y":1344},[208,3514,3516],{"dataMmlNode":218,"transform":3515},"translate(4086.9,0)",[220,3517],{"dataC":1089,"xLinkHref":3518},"#MJX-35-TEX-N-3D",[208,3520,3522,3527,3532],{"dataMmlNode":1324,"transform":3521},"translate(5142.6,0)",[208,3523,3524],{"dataMmlNode":1103,"transform":1328},[220,3525],{"dataC":1127,"xLinkHref":3526},"#MJX-35-TEX-N-31",[208,3528,3529],{"dataMmlNode":1103,"transform":1334},[220,3530],{"dataC":1157,"xLinkHref":3531},"#MJX-35-TEX-N-32",[1339,3533],{"width":1341,"height":1342,"x":1343,"y":1344},[208,3535,3537],{"dataMmlNode":218,"transform":3536},"translate(6304.9,0)",[220,3538],{"dataC":3035,"xLinkHref":3539},"#MJX-35-TEX-N-22C5",[208,3541,3543],{"dataMmlNode":1103,"transform":3542},"translate(6805.1,0)",[220,3544],{"dataC":1157,"xLinkHref":3531},[208,3546,3548,3618],{"dataMmlNode":630,"transform":3547},"translate(7305.1,0)",[208,3549,3550,3555,3559,3563,3583,3587,3592,3613],{"dataMmlNode":1821},[208,3551,3552],{"dataMmlNode":218,"transform":2463},[220,3553],{"dataC":222,"xLinkHref":3554},"#MJX-35-TEX-LO-28",[208,3556,3557],{"dataMmlNode":226,"transform":2469},[220,3558],{"dataC":876,"xLinkHref":3480},[208,3560,3561],{"dataMmlNode":218,"transform":2475},[220,3562],{"dataC":222,"xLinkHref":3459},[208,3564,3565,3569],{"dataMmlNode":630,"transform":2480},[208,3566,3567],{"dataMmlNode":226},[220,3568],{"dataC":230,"xLinkHref":3491},[208,3570,3571,3575,3579],{"dataMmlNode":269,"transform":1387,"dataMjxTexclass":270},[208,3572,3573],{"dataMmlNode":218},[220,3574],{"dataC":222,"xLinkHref":3459},[208,3576,3577],{"dataMmlNode":226,"transform":227},[220,3578],{"dataC":251,"xLinkHref":3502},[208,3580,3581],{"dataMmlNode":218,"transform":651},[220,3582],{"dataC":265,"xLinkHref":3469},[208,3584,3585],{"dataMmlNode":218,"transform":2502},[220,3586],{"dataC":265,"xLinkHref":3469},[208,3588,3589],{"dataMmlNode":218,"transform":2507},[220,3590],{"dataC":237,"xLinkHref":3591},"#MJX-35-TEX-N-2212",[208,3593,3594,3599],{"dataMmlNode":630,"transform":2513},[208,3595,3596],{"dataMmlNode":226},[220,3597],{"dataC":694,"xLinkHref":3598},"#MJX-35-TEX-I-1D466",[208,3600,3601,3605,3609],{"dataMmlNode":269,"transform":1421,"dataMjxTexclass":270},[208,3602,3603],{"dataMmlNode":218},[220,3604],{"dataC":222,"xLinkHref":3459},[208,3606,3607],{"dataMmlNode":226,"transform":227},[220,3608],{"dataC":251,"xLinkHref":3502},[208,3610,3611],{"dataMmlNode":218,"transform":651},[220,3612],{"dataC":265,"xLinkHref":3469},[208,3614,3615],{"dataMmlNode":218,"transform":2535},[220,3616],{"dataC":265,"xLinkHref":3617},"#MJX-35-TEX-LO-29",[208,3619,3620,3624,3628],{"dataMmlNode":269,"transform":2541,"dataMjxTexclass":270},[208,3621,3622],{"dataMmlNode":1103},[220,3623],{"dataC":1157,"xLinkHref":3531},[208,3625,3626],{"dataMmlNode":218,"transform":3275},[220,3627],{"dataC":237,"xLinkHref":3591},[208,3629,3631],{"dataMmlNode":1103,"transform":3630},"translate(1278,0)",[220,3632],{"dataC":1127,"xLinkHref":3526},[151,3634,3636],{"className":3635,"jax":155,"display":1017},[154],[157,3637,3640,3681],{"style":3383,"xmlns":160,"width":3638,"height":3385,"role":163,"focusable":164,"viewBox":3639,"xmlnsXLink":166},"26.537ex","0 -1460 11729.2 2489.3",[168,3641,3642,3645,3648,3651,3654,3657,3660,3663,3666,3669,3672,3675,3678],{},[171,3643],{"id":3644,"d":1750},"MJX-36-TEX-I-1D715",[171,3646],{"id":3647,"d":1250},"MJX-36-TEX-I-1D43D",[171,3649],{"id":3650,"d":174},"MJX-36-TEX-N-28",[171,3652],{"id":3653,"d":1043},"MJX-36-TEX-I-1D703",[171,3655],{"id":3656,"d":198},"MJX-36-TEX-N-29",[171,3658],{"id":3659,"d":858},"MJX-36-TEX-I-210E",[171,3661],{"id":3662,"d":178},"MJX-36-TEX-I-1D465",[171,3664],{"id":3665,"d":190},"MJX-36-TEX-I-1D456",[171,3667],{"id":3668,"d":1039},"MJX-36-TEX-N-3D",[171,3670],{"id":3671,"d":2384},"MJX-36-TEX-LO-28",[171,3673],{"id":3674,"d":182},"MJX-36-TEX-N-2212",[171,3676],{"id":3677,"d":674},"MJX-36-TEX-I-1D466",[171,3679],{"id":3680,"d":2400},"MJX-36-TEX-LO-29",[208,3682,3683],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,3684,3685,3757,3762],{"dataMmlNode":215},[208,3686,3687,3714,3755],{"dataMmlNode":1324},[208,3688,3689,3694,3699,3704,3709],{"dataMmlNode":1821,"transform":2961},[208,3690,3691],{"dataMmlNode":226},[220,3692],{"dataC":1827,"xLinkHref":3693},"#MJX-36-TEX-I-1D715",[208,3695,3696],{"dataMmlNode":226,"transform":1831},[220,3697],{"dataC":1296,"xLinkHref":3698},"#MJX-36-TEX-I-1D43D",[208,3700,3701],{"dataMmlNode":218,"transform":1837},[220,3702],{"dataC":222,"xLinkHref":3703},"#MJX-36-TEX-N-28",[208,3705,3706],{"dataMmlNode":226,"transform":1843},[220,3707],{"dataC":1099,"xLinkHref":3708},"#MJX-36-TEX-I-1D703",[208,3710,3711],{"dataMmlNode":218,"transform":1848},[220,3712],{"dataC":265,"xLinkHref":3713},"#MJX-36-TEX-N-29",[208,3715,3716,3720,3725,3729,3751],{"dataMmlNode":1821,"transform":2984},[208,3717,3718],{"dataMmlNode":226},[220,3719],{"dataC":1827,"xLinkHref":3693},[208,3721,3722],{"dataMmlNode":226,"transform":1831},[220,3723],{"dataC":876,"xLinkHref":3724},"#MJX-36-TEX-I-210E",[208,3726,3727],{"dataMmlNode":218,"transform":2996},[220,3728],{"dataC":222,"xLinkHref":3703},[208,3730,3731,3736],{"dataMmlNode":630,"transform":3001},[208,3732,3733],{"dataMmlNode":226},[220,3734],{"dataC":230,"xLinkHref":3735},"#MJX-36-TEX-I-1D465",[208,3737,3738,3742,3747],{"dataMmlNode":269,"transform":3009,"dataMjxTexclass":270},[208,3739,3740],{"dataMmlNode":218},[220,3741],{"dataC":222,"xLinkHref":3703},[208,3743,3744],{"dataMmlNode":226,"transform":227},[220,3745],{"dataC":251,"xLinkHref":3746},"#MJX-36-TEX-I-1D456",[208,3748,3749],{"dataMmlNode":218,"transform":651},[220,3750],{"dataC":265,"xLinkHref":3713},[208,3752,3753],{"dataMmlNode":218,"transform":3024},[220,3754],{"dataC":265,"xLinkHref":3713},[1339,3756],{"width":3029,"height":1342,"x":1343,"y":1344},[208,3758,3759],{"dataMmlNode":218,"transform":3515},[220,3760],{"dataC":1089,"xLinkHref":3761},"#MJX-36-TEX-N-3D",[208,3763,3764,3769,3773,3777,3797,3801,3806,3827],{"dataMmlNode":1821,"transform":3521},[208,3765,3766],{"dataMmlNode":218,"transform":2463},[220,3767],{"dataC":222,"xLinkHref":3768},"#MJX-36-TEX-LO-28",[208,3770,3771],{"dataMmlNode":226,"transform":2469},[220,3772],{"dataC":876,"xLinkHref":3724},[208,3774,3775],{"dataMmlNode":218,"transform":2475},[220,3776],{"dataC":222,"xLinkHref":3703},[208,3778,3779,3783],{"dataMmlNode":630,"transform":2480},[208,3780,3781],{"dataMmlNode":226},[220,3782],{"dataC":230,"xLinkHref":3735},[208,3784,3785,3789,3793],{"dataMmlNode":269,"transform":1387,"dataMjxTexclass":270},[208,3786,3787],{"dataMmlNode":218},[220,3788],{"dataC":222,"xLinkHref":3703},[208,3790,3791],{"dataMmlNode":226,"transform":227},[220,3792],{"dataC":251,"xLinkHref":3746},[208,3794,3795],{"dataMmlNode":218,"transform":651},[220,3796],{"dataC":265,"xLinkHref":3713},[208,3798,3799],{"dataMmlNode":218,"transform":2502},[220,3800],{"dataC":265,"xLinkHref":3713},[208,3802,3803],{"dataMmlNode":218,"transform":2507},[220,3804],{"dataC":237,"xLinkHref":3805},"#MJX-36-TEX-N-2212",[208,3807,3808,3813],{"dataMmlNode":630,"transform":2513},[208,3809,3810],{"dataMmlNode":226},[220,3811],{"dataC":694,"xLinkHref":3812},"#MJX-36-TEX-I-1D466",[208,3814,3815,3819,3823],{"dataMmlNode":269,"transform":1421,"dataMjxTexclass":270},[208,3816,3817],{"dataMmlNode":218},[220,3818],{"dataC":222,"xLinkHref":3703},[208,3820,3821],{"dataMmlNode":226,"transform":227},[220,3822],{"dataC":251,"xLinkHref":3746},[208,3824,3825],{"dataMmlNode":218,"transform":651},[220,3826],{"dataC":265,"xLinkHref":3713},[208,3828,3829],{"dataMmlNode":218,"transform":2535},[220,3830],{"dataC":265,"xLinkHref":3831},"#MJX-36-TEX-LO-29",[16,3833,3834,1498,3837,3873,3874,3939,3940,3969,3970,3992,3993,4049],{},[20,3835,3836],{},"Part 2: How does the prediction change with respect to the weight",[20,3838,3839,3872],{},[151,3840,3842],{"className":3841,"jax":155},[154],[157,3843,3848,3856],{"style":3844,"xmlns":160,"width":3845,"height":3846,"role":163,"focusable":164,"viewBox":3847,"xmlnsXLink":166},"vertical-align: -0.666ex;","1.908ex","2.261ex","0 -705 843.3 999.2",[168,3849,3850,3853],{},[171,3851],{"id":3852,"d":1043},"MJX-37-TEX-I-1D703",[171,3854],{"id":3855,"d":1732},"MJX-37-TEX-I-1D457",[208,3857,3858],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,3859,3860],{"dataMmlNode":215},[208,3861,3862,3867],{"dataMmlNode":1093},[208,3863,3864],{"dataMmlNode":226},[220,3865],{"dataC":1099,"xLinkHref":3866},"#MJX-37-TEX-I-1D703",[208,3868,3869],{"dataMmlNode":226,"transform":1104},[220,3870],{"dataC":1775,"xLinkHref":3871},"#MJX-37-TEX-I-1D457","?","\nLooking at the hypothesis ",[151,3875,3877],{"className":3876,"jax":155},[154],[157,3878,3881,3898],{"style":159,"xmlns":160,"width":3879,"height":725,"role":163,"focusable":164,"viewBox":3880,"xmlnsXLink":166},"6.342ex","0 -893.3 2803.1 1143.3",[168,3882,3883,3886,3889,3892,3895],{},[171,3884],{"id":3885,"d":858},"MJX-38-TEX-I-210E",[171,3887],{"id":3888,"d":174},"MJX-38-TEX-N-28",[171,3890],{"id":3891,"d":178},"MJX-38-TEX-I-1D465",[171,3893],{"id":3894,"d":190},"MJX-38-TEX-I-1D456",[171,3896],{"id":3897,"d":198},"MJX-38-TEX-N-29",[208,3899,3900],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,3901,3902,3907,3912,3935],{"dataMmlNode":215},[208,3903,3904],{"dataMmlNode":226},[220,3905],{"dataC":876,"xLinkHref":3906},"#MJX-38-TEX-I-210E",[208,3908,3909],{"dataMmlNode":218,"transform":880},[220,3910],{"dataC":222,"xLinkHref":3911},"#MJX-38-TEX-N-28",[208,3913,3914,3919],{"dataMmlNode":630,"transform":886},[208,3915,3916],{"dataMmlNode":226},[220,3917],{"dataC":230,"xLinkHref":3918},"#MJX-38-TEX-I-1D465",[208,3920,3921,3925,3930],{"dataMmlNode":269,"transform":638,"dataMjxTexclass":270},[208,3922,3923],{"dataMmlNode":218},[220,3924],{"dataC":222,"xLinkHref":3911},[208,3926,3927],{"dataMmlNode":226,"transform":227},[220,3928],{"dataC":251,"xLinkHref":3929},"#MJX-38-TEX-I-1D456",[208,3931,3932],{"dataMmlNode":218,"transform":651},[220,3933],{"dataC":265,"xLinkHref":3934},"#MJX-38-TEX-N-29",[208,3936,3937],{"dataMmlNode":218,"transform":2639},[220,3938],{"dataC":265,"xLinkHref":3934},", if we take the derivative with respect to ",[151,3941,3943],{"className":3942,"jax":155},[154],[157,3944,3945,3953],{"style":3844,"xmlns":160,"width":3845,"height":3846,"role":163,"focusable":164,"viewBox":3847,"xmlnsXLink":166},[168,3946,3947,3950],{},[171,3948],{"id":3949,"d":1043},"MJX-39-TEX-I-1D703",[171,3951],{"id":3952,"d":1732},"MJX-39-TEX-I-1D457",[208,3954,3955],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,3956,3957],{"dataMmlNode":215},[208,3958,3959,3964],{"dataMmlNode":1093},[208,3960,3961],{"dataMmlNode":226},[220,3962],{"dataC":1099,"xLinkHref":3963},"#MJX-39-TEX-I-1D703",[208,3965,3966],{"dataMmlNode":226,"transform":1104},[220,3967],{"dataC":1775,"xLinkHref":3968},"#MJX-39-TEX-I-1D457",", all other terms become ",[151,3971,3973],{"className":3972,"jax":155},[154],[157,3974,3978,3983],{"style":3975,"xmlns":160,"width":3337,"height":3976,"role":163,"focusable":164,"viewBox":3977,"xmlnsXLink":166},"vertical-align: -0.05ex;","1.557ex","0 -666 500 688",[168,3979,3980],{},[171,3981],{"id":3982,"d":1047},"MJX-40-TEX-N-30",[208,3984,3985],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,3986,3987],{"dataMmlNode":215},[208,3988,3989],{"dataMmlNode":1103},[220,3990],{"dataC":1107,"xLinkHref":3991},"#MJX-40-TEX-N-30",", leaving only the feature ",[151,3994,3996],{"className":3995,"jax":155},[154],[157,3997,3999,4016],{"style":1965,"xmlns":160,"width":607,"height":1967,"role":163,"focusable":164,"viewBox":3998,"xmlnsXLink":166},"0 -1060.7 1449.1 1498.8",[168,4000,4001,4004,4007,4010,4013],{},[171,4002],{"id":4003,"d":178},"MJX-41-TEX-I-1D465",[171,4005],{"id":4006,"d":174},"MJX-41-TEX-N-28",[171,4008],{"id":4009,"d":190},"MJX-41-TEX-I-1D456",[171,4011],{"id":4012,"d":198},"MJX-41-TEX-N-29",[171,4014],{"id":4015,"d":1732},"MJX-41-TEX-I-1D457",[208,4017,4018],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,4019,4020],{"dataMmlNode":215},[208,4021,4022,4027,4044],{"dataMmlNode":2138},[208,4023,4024],{"dataMmlNode":226},[220,4025],{"dataC":230,"xLinkHref":4026},"#MJX-41-TEX-I-1D465",[208,4028,4029,4034,4039],{"dataMmlNode":269,"transform":2146,"dataMjxTexclass":270},[208,4030,4031],{"dataMmlNode":218},[220,4032],{"dataC":222,"xLinkHref":4033},"#MJX-41-TEX-N-28",[208,4035,4036],{"dataMmlNode":226,"transform":227},[220,4037],{"dataC":251,"xLinkHref":4038},"#MJX-41-TEX-I-1D456",[208,4040,4041],{"dataMmlNode":218,"transform":651},[220,4042],{"dataC":265,"xLinkHref":4043},"#MJX-41-TEX-N-29",[208,4045,4046],{"dataMmlNode":226,"transform":2161},[220,4047],{"dataC":1775,"xLinkHref":4048},"#MJX-41-TEX-I-1D457",":",[151,4051,4053],{"className":4052,"jax":155,"display":1017},[154],[157,4054,4058,4087],{"style":1720,"xmlns":160,"width":4055,"height":4056,"role":163,"focusable":164,"viewBox":4057,"xmlnsXLink":166},"14.913ex","5.845ex","0 -1603.3 6591.7 2583.6",[168,4059,4060,4063,4066,4069,4072,4075,4078,4081,4084],{},[171,4061],{"id":4062,"d":1750},"MJX-42-TEX-I-1D715",[171,4064],{"id":4065,"d":858},"MJX-42-TEX-I-210E",[171,4067],{"id":4068,"d":174},"MJX-42-TEX-N-28",[171,4070],{"id":4071,"d":178},"MJX-42-TEX-I-1D465",[171,4073],{"id":4074,"d":190},"MJX-42-TEX-I-1D456",[171,4076],{"id":4077,"d":198},"MJX-42-TEX-N-29",[171,4079],{"id":4080,"d":1043},"MJX-42-TEX-I-1D703",[171,4082],{"id":4083,"d":1732},"MJX-42-TEX-I-1D457",[171,4085],{"id":4086,"d":1039},"MJX-42-TEX-N-3D",[208,4088,4089],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,4090,4091,4157,4162],{"dataMmlNode":215},[208,4092,4093,4137,4155],{"dataMmlNode":1324},[208,4094,4095,4100,4105,4110,4133],{"dataMmlNode":1821,"transform":1822},[208,4096,4097],{"dataMmlNode":226},[220,4098],{"dataC":1827,"xLinkHref":4099},"#MJX-42-TEX-I-1D715",[208,4101,4102],{"dataMmlNode":226,"transform":1831},[220,4103],{"dataC":876,"xLinkHref":4104},"#MJX-42-TEX-I-210E",[208,4106,4107],{"dataMmlNode":218,"transform":2996},[220,4108],{"dataC":222,"xLinkHref":4109},"#MJX-42-TEX-N-28",[208,4111,4112,4117],{"dataMmlNode":630,"transform":3001},[208,4113,4114],{"dataMmlNode":226},[220,4115],{"dataC":230,"xLinkHref":4116},"#MJX-42-TEX-I-1D465",[208,4118,4119,4123,4128],{"dataMmlNode":269,"transform":638,"dataMjxTexclass":270},[208,4120,4121],{"dataMmlNode":218},[220,4122],{"dataC":222,"xLinkHref":4109},[208,4124,4125],{"dataMmlNode":226,"transform":227},[220,4126],{"dataC":251,"xLinkHref":4127},"#MJX-42-TEX-I-1D456",[208,4129,4130],{"dataMmlNode":218,"transform":651},[220,4131],{"dataC":265,"xLinkHref":4132},"#MJX-42-TEX-N-29",[208,4134,4135],{"dataMmlNode":218,"transform":3024},[220,4136],{"dataC":265,"xLinkHref":4132},[208,4138,4139,4143],{"dataMmlNode":1821,"transform":3080},[208,4140,4141],{"dataMmlNode":226},[220,4142],{"dataC":1827,"xLinkHref":4099},[208,4144,4145,4150],{"dataMmlNode":1093,"transform":1831},[208,4146,4147],{"dataMmlNode":226},[220,4148],{"dataC":1099,"xLinkHref":4149},"#MJX-42-TEX-I-1D703",[208,4151,4152],{"dataMmlNode":226,"transform":1104},[220,4153],{"dataC":1775,"xLinkHref":4154},"#MJX-42-TEX-I-1D457",[1339,4156],{"width":3029,"height":1342,"x":1343,"y":1344},[208,4158,4159],{"dataMmlNode":218,"transform":3515},[220,4160],{"dataC":1089,"xLinkHref":4161},"#MJX-42-TEX-N-3D",[208,4163,4164,4168,4182],{"dataMmlNode":2138,"transform":3521},[208,4165,4166],{"dataMmlNode":226},[220,4167],{"dataC":230,"xLinkHref":4116},[208,4169,4170,4174,4178],{"dataMmlNode":269,"transform":2146,"dataMjxTexclass":270},[208,4171,4172],{"dataMmlNode":218},[220,4173],{"dataC":222,"xLinkHref":4109},[208,4175,4176],{"dataMmlNode":226,"transform":227},[220,4177],{"dataC":251,"xLinkHref":4127},[208,4179,4180],{"dataMmlNode":218,"transform":651},[220,4181],{"dataC":265,"xLinkHref":4132},[208,4183,4184],{"dataMmlNode":226,"transform":2161},[220,4185],{"dataC":1775,"xLinkHref":4154},[2814,4187,4189],{"id":4188},"combine-the-chain-rule","Combine the Chain Rule",[16,4191,4192],{},"Now, we multiply Part 1 and Part 2 together and place them back inside our summation:",[151,4194,4196],{"className":4195,"jax":155,"display":1017},[154],[157,4197,4201,4259],{"style":1241,"xmlns":160,"width":4198,"height":4199,"role":163,"focusable":164,"viewBox":4200,"xmlnsXLink":166},"35.141ex","6ex","0 -1460 15532.4 2652.2",[168,4202,4203,4206,4209,4212,4215,4218,4221,4224,4227,4230,4234,4237,4240,4243,4246,4249,4252,4255],{},[171,4204],{"id":4205,"d":1750},"MJX-43-TEX-I-1D715",[171,4207],{"id":4208,"d":1250},"MJX-43-TEX-I-1D43D",[171,4210],{"id":4211,"d":174},"MJX-43-TEX-N-28",[171,4213],{"id":4214,"d":1043},"MJX-43-TEX-I-1D703",[171,4216],{"id":4217,"d":198},"MJX-43-TEX-N-29",[171,4219],{"id":4220,"d":1732},"MJX-43-TEX-I-1D457",[171,4222],{"id":4223,"d":1039},"MJX-43-TEX-N-3D",[171,4225],{"id":4226,"d":1272},"MJX-43-TEX-LO-2211",[171,4228],{"id":4229,"d":190},"MJX-43-TEX-I-1D456",[171,4231],{"id":4232,"d":4233},"MJX-43-TEX-LO-5B","M224 -649V1150H455V1099H275V-598H455V-649H224Z",[171,4235],{"id":4236,"d":2384},"MJX-43-TEX-LO-28",[171,4238],{"id":4239,"d":858},"MJX-43-TEX-I-210E",[171,4241],{"id":4242,"d":178},"MJX-43-TEX-I-1D465",[171,4244],{"id":4245,"d":182},"MJX-43-TEX-N-2212",[171,4247],{"id":4248,"d":674},"MJX-43-TEX-I-1D466",[171,4250],{"id":4251,"d":2400},"MJX-43-TEX-LO-29",[171,4253],{"id":4254,"d":2872},"MJX-43-TEX-N-22C5",[171,4256],{"id":4257,"d":4258},"MJX-43-TEX-LO-5D","M16 1099V1150H247V-649H16V-598H196V1099H16Z",[208,4260,4261],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,4262,4263,4311,4316,4328],{"dataMmlNode":215},[208,4264,4265,4292,4309],{"dataMmlNode":1324},[208,4266,4267,4272,4277,4282,4287],{"dataMmlNode":1821,"transform":1822},[208,4268,4269],{"dataMmlNode":226},[220,4270],{"dataC":1827,"xLinkHref":4271},"#MJX-43-TEX-I-1D715",[208,4273,4274],{"dataMmlNode":226,"transform":1831},[220,4275],{"dataC":1296,"xLinkHref":4276},"#MJX-43-TEX-I-1D43D",[208,4278,4279],{"dataMmlNode":218,"transform":1837},[220,4280],{"dataC":222,"xLinkHref":4281},"#MJX-43-TEX-N-28",[208,4283,4284],{"dataMmlNode":226,"transform":1843},[220,4285],{"dataC":1099,"xLinkHref":4286},"#MJX-43-TEX-I-1D703",[208,4288,4289],{"dataMmlNode":218,"transform":1848},[220,4290],{"dataC":265,"xLinkHref":4291},"#MJX-43-TEX-N-29",[208,4293,4294,4298],{"dataMmlNode":1821,"transform":1854},[208,4295,4296],{"dataMmlNode":226},[220,4297],{"dataC":1827,"xLinkHref":4271},[208,4299,4300,4304],{"dataMmlNode":1093,"transform":1831},[208,4301,4302],{"dataMmlNode":226},[220,4303],{"dataC":1099,"xLinkHref":4286},[208,4305,4306],{"dataMmlNode":226,"transform":1104},[220,4307],{"dataC":1775,"xLinkHref":4308},"#MJX-43-TEX-I-1D457",[1339,4310],{"width":1871,"height":1342,"x":1343,"y":1344},[208,4312,4313],{"dataMmlNode":218,"transform":2931},[220,4314],{"dataC":1089,"xLinkHref":4315},"#MJX-43-TEX-N-3D",[208,4317,4318,4323],{"dataMmlNode":1347,"transform":2937},[208,4319,4320],{"dataMmlNode":218},[220,4321],{"dataC":1353,"xLinkHref":4322},"#MJX-43-TEX-LO-2211",[208,4324,4325],{"dataMmlNode":226,"transform":1357},[220,4326],{"dataC":251,"xLinkHref":4327},"#MJX-43-TEX-I-1D456",[208,4329,4330,4336,4409,4415,4440],{"dataMmlNode":1821,"transform":2950},[208,4331,4332],{"dataMmlNode":218,"transform":2463},[220,4333],{"dataC":4334,"xLinkHref":4335},"5B","#MJX-43-TEX-LO-5B",[208,4337,4339,4344,4349,4353,4374,4378,4383,4404],{"dataMmlNode":1821,"transform":4338},"translate(472,0)",[208,4340,4341],{"dataMmlNode":218,"transform":2463},[220,4342],{"dataC":222,"xLinkHref":4343},"#MJX-43-TEX-LO-28",[208,4345,4346],{"dataMmlNode":226,"transform":2469},[220,4347],{"dataC":876,"xLinkHref":4348},"#MJX-43-TEX-I-210E",[208,4350,4351],{"dataMmlNode":218,"transform":2475},[220,4352],{"dataC":222,"xLinkHref":4281},[208,4354,4355,4360],{"dataMmlNode":630,"transform":2480},[208,4356,4357],{"dataMmlNode":226},[220,4358],{"dataC":230,"xLinkHref":4359},"#MJX-43-TEX-I-1D465",[208,4361,4362,4366,4370],{"dataMmlNode":269,"transform":1387,"dataMjxTexclass":270},[208,4363,4364],{"dataMmlNode":218},[220,4365],{"dataC":222,"xLinkHref":4281},[208,4367,4368],{"dataMmlNode":226,"transform":227},[220,4369],{"dataC":251,"xLinkHref":4327},[208,4371,4372],{"dataMmlNode":218,"transform":651},[220,4373],{"dataC":265,"xLinkHref":4291},[208,4375,4376],{"dataMmlNode":218,"transform":2502},[220,4377],{"dataC":265,"xLinkHref":4291},[208,4379,4380],{"dataMmlNode":218,"transform":2507},[220,4381],{"dataC":237,"xLinkHref":4382},"#MJX-43-TEX-N-2212",[208,4384,4385,4390],{"dataMmlNode":630,"transform":2513},[208,4386,4387],{"dataMmlNode":226},[220,4388],{"dataC":694,"xLinkHref":4389},"#MJX-43-TEX-I-1D466",[208,4391,4392,4396,4400],{"dataMmlNode":269,"transform":1421,"dataMjxTexclass":270},[208,4393,4394],{"dataMmlNode":218},[220,4395],{"dataC":222,"xLinkHref":4281},[208,4397,4398],{"dataMmlNode":226,"transform":227},[220,4399],{"dataC":251,"xLinkHref":4327},[208,4401,4402],{"dataMmlNode":218,"transform":651},[220,4403],{"dataC":265,"xLinkHref":4291},[208,4405,4406],{"dataMmlNode":218,"transform":2535},[220,4407],{"dataC":265,"xLinkHref":4408},"#MJX-43-TEX-LO-29",[208,4410,4412],{"dataMmlNode":218,"transform":4411},"translate(7280.8,0)",[220,4413],{"dataC":3035,"xLinkHref":4414},"#MJX-43-TEX-N-22C5",[208,4416,4418,4422,4436],{"dataMmlNode":2138,"transform":4417},"translate(7781.1,0)",[208,4419,4420],{"dataMmlNode":226},[220,4421],{"dataC":230,"xLinkHref":4359},[208,4423,4424,4428,4432],{"dataMmlNode":269,"transform":2146,"dataMjxTexclass":270},[208,4425,4426],{"dataMmlNode":218},[220,4427],{"dataC":222,"xLinkHref":4281},[208,4429,4430],{"dataMmlNode":226,"transform":227},[220,4431],{"dataC":251,"xLinkHref":4327},[208,4433,4434],{"dataMmlNode":218,"transform":651},[220,4435],{"dataC":265,"xLinkHref":4291},[208,4437,4438],{"dataMmlNode":226,"transform":2161},[220,4439],{"dataC":1775,"xLinkHref":4308},[208,4441,4443],{"dataMmlNode":218,"transform":4442},"translate(9230.1,0) translate(0 -0.5)",[220,4444],{"dataC":4445,"xLinkHref":4446},"5D","#MJX-43-TEX-LO-5D",[2814,4448,4450],{"id":4449},"the-final-update-rules","The Final Update Rules",[16,4452,4453],{},"Substitute the combined derivative back into the original gradient descent update rule:",[151,4455,4457],{"className":4456,"jax":155,"display":1017},[154],[157,4458,4462,4518],{"style":1241,"xmlns":160,"width":4459,"height":4460,"role":163,"focusable":164,"viewBox":4461,"xmlnsXLink":166},"37.647ex","5.298ex","0 -1149.5 16640.1 2341.7",[168,4463,4464,4467,4470,4473,4476,4479,4482,4485,4488,4491,4494,4497,4500,4503,4506,4509,4512,4515],{},[171,4465],{"id":4466,"d":1043},"MJX-44-TEX-I-1D703",[171,4468],{"id":4469,"d":1732},"MJX-44-TEX-I-1D457",[171,4471],{"id":4472,"d":1736},"MJX-44-TEX-N-3A",[171,4474],{"id":4475,"d":1039},"MJX-44-TEX-N-3D",[171,4477],{"id":4478,"d":182},"MJX-44-TEX-N-2212",[171,4480],{"id":4481,"d":1746},"MJX-44-TEX-I-1D6FC",[171,4483],{"id":4484,"d":1272},"MJX-44-TEX-LO-2211",[171,4486],{"id":4487,"d":190},"MJX-44-TEX-I-1D456",[171,4489],{"id":4490,"d":4233},"MJX-44-TEX-LO-5B",[171,4492],{"id":4493,"d":2384},"MJX-44-TEX-LO-28",[171,4495],{"id":4496,"d":858},"MJX-44-TEX-I-210E",[171,4498],{"id":4499,"d":174},"MJX-44-TEX-N-28",[171,4501],{"id":4502,"d":178},"MJX-44-TEX-I-1D465",[171,4504],{"id":4505,"d":198},"MJX-44-TEX-N-29",[171,4507],{"id":4508,"d":674},"MJX-44-TEX-I-1D466",[171,4510],{"id":4511,"d":2400},"MJX-44-TEX-LO-29",[171,4513],{"id":4514,"d":2872},"MJX-44-TEX-N-22C5",[171,4516],{"id":4517,"d":4258},"MJX-44-TEX-LO-5D",[208,4519,4520],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,4521,4522,4534,4546,4556,4561,4566,4579],{"dataMmlNode":215},[208,4523,4524,4529],{"dataMmlNode":1093},[208,4525,4526],{"dataMmlNode":226},[220,4527],{"dataC":1099,"xLinkHref":4528},"#MJX-44-TEX-I-1D703",[208,4530,4531],{"dataMmlNode":226,"transform":1104},[220,4532],{"dataC":1775,"xLinkHref":4533},"#MJX-44-TEX-I-1D457",[208,4535,4536,4541],{"dataMmlNode":218,"transform":1779},[208,4537,4538],{"dataMmlNode":980},[220,4539],{"dataC":1784,"xLinkHref":4540},"#MJX-44-TEX-N-3A",[208,4542,4543],{"dataMmlNode":980,"transform":1788},[220,4544],{"dataC":1089,"xLinkHref":4545},"#MJX-44-TEX-N-3D",[208,4547,4548,4552],{"dataMmlNode":1093,"transform":1794},[208,4549,4550],{"dataMmlNode":226},[220,4551],{"dataC":1099,"xLinkHref":4528},[208,4553,4554],{"dataMmlNode":226,"transform":1104},[220,4555],{"dataC":1775,"xLinkHref":4533},[208,4557,4558],{"dataMmlNode":218,"transform":1805},[220,4559],{"dataC":237,"xLinkHref":4560},"#MJX-44-TEX-N-2212",[208,4562,4563],{"dataMmlNode":226,"transform":1811},[220,4564],{"dataC":1814,"xLinkHref":4565},"#MJX-44-TEX-I-1D6FC",[208,4567,4569,4574],{"dataMmlNode":1347,"transform":4568},"translate(5327.3,0)",[208,4570,4571],{"dataMmlNode":218},[220,4572],{"dataC":1353,"xLinkHref":4573},"#MJX-44-TEX-LO-2211",[208,4575,4576],{"dataMmlNode":226,"transform":1357},[220,4577],{"dataC":251,"xLinkHref":4578},"#MJX-44-TEX-I-1D456",[208,4580,4582,4587,4660,4665,4689],{"dataMmlNode":1821,"transform":4581},"translate(6938,0)",[208,4583,4584],{"dataMmlNode":218,"transform":2463},[220,4585],{"dataC":4334,"xLinkHref":4586},"#MJX-44-TEX-LO-5B",[208,4588,4589,4594,4599,4604,4626,4630,4634,4655],{"dataMmlNode":1821,"transform":4338},[208,4590,4591],{"dataMmlNode":218,"transform":2463},[220,4592],{"dataC":222,"xLinkHref":4593},"#MJX-44-TEX-LO-28",[208,4595,4596],{"dataMmlNode":226,"transform":2469},[220,4597],{"dataC":876,"xLinkHref":4598},"#MJX-44-TEX-I-210E",[208,4600,4601],{"dataMmlNode":218,"transform":2475},[220,4602],{"dataC":222,"xLinkHref":4603},"#MJX-44-TEX-N-28",[208,4605,4606,4611],{"dataMmlNode":630,"transform":2480},[208,4607,4608],{"dataMmlNode":226},[220,4609],{"dataC":230,"xLinkHref":4610},"#MJX-44-TEX-I-1D465",[208,4612,4613,4617,4621],{"dataMmlNode":269,"transform":1387,"dataMjxTexclass":270},[208,4614,4615],{"dataMmlNode":218},[220,4616],{"dataC":222,"xLinkHref":4603},[208,4618,4619],{"dataMmlNode":226,"transform":227},[220,4620],{"dataC":251,"xLinkHref":4578},[208,4622,4623],{"dataMmlNode":218,"transform":651},[220,4624],{"dataC":265,"xLinkHref":4625},"#MJX-44-TEX-N-29",[208,4627,4628],{"dataMmlNode":218,"transform":2502},[220,4629],{"dataC":265,"xLinkHref":4625},[208,4631,4632],{"dataMmlNode":218,"transform":2507},[220,4633],{"dataC":237,"xLinkHref":4560},[208,4635,4636,4641],{"dataMmlNode":630,"transform":2513},[208,4637,4638],{"dataMmlNode":226},[220,4639],{"dataC":694,"xLinkHref":4640},"#MJX-44-TEX-I-1D466",[208,4642,4643,4647,4651],{"dataMmlNode":269,"transform":1421,"dataMjxTexclass":270},[208,4644,4645],{"dataMmlNode":218},[220,4646],{"dataC":222,"xLinkHref":4603},[208,4648,4649],{"dataMmlNode":226,"transform":227},[220,4650],{"dataC":251,"xLinkHref":4578},[208,4652,4653],{"dataMmlNode":218,"transform":651},[220,4654],{"dataC":265,"xLinkHref":4625},[208,4656,4657],{"dataMmlNode":218,"transform":2535},[220,4658],{"dataC":265,"xLinkHref":4659},"#MJX-44-TEX-LO-29",[208,4661,4662],{"dataMmlNode":218,"transform":4411},[220,4663],{"dataC":3035,"xLinkHref":4664},"#MJX-44-TEX-N-22C5",[208,4666,4667,4671,4685],{"dataMmlNode":2138,"transform":4417},[208,4668,4669],{"dataMmlNode":226},[220,4670],{"dataC":230,"xLinkHref":4610},[208,4672,4673,4677,4681],{"dataMmlNode":269,"transform":2146,"dataMjxTexclass":270},[208,4674,4675],{"dataMmlNode":218},[220,4676],{"dataC":222,"xLinkHref":4603},[208,4678,4679],{"dataMmlNode":226,"transform":227},[220,4680],{"dataC":251,"xLinkHref":4578},[208,4682,4683],{"dataMmlNode":218,"transform":651},[220,4684],{"dataC":265,"xLinkHref":4625},[208,4686,4687],{"dataMmlNode":226,"transform":2161},[220,4688],{"dataC":1775,"xLinkHref":4533},[208,4690,4691],{"dataMmlNode":218,"transform":4442},[220,4692],{"dataC":4445,"xLinkHref":4693},"#MJX-44-TEX-LO-5D",[16,4695,4696,4697,4771,4772,1542],{},"To get the most elegant form, distribute the negative sign to flip the inner terms ",[151,4698,4700],{"className":4699,"jax":155},[154],[157,4701,4704,4724],{"style":159,"xmlns":160,"width":4702,"height":162,"role":163,"focusable":164,"viewBox":4703,"xmlnsXLink":166},"9.992ex","0 -750 4416.4 1000",[168,4705,4706,4709,4712,4715,4718,4721],{},[171,4707],{"id":4708,"d":174},"MJX-45-TEX-N-28",[171,4710],{"id":4711,"d":858},"MJX-45-TEX-I-210E",[171,4713],{"id":4714,"d":178},"MJX-45-TEX-I-1D465",[171,4716],{"id":4717,"d":198},"MJX-45-TEX-N-29",[171,4719],{"id":4720,"d":182},"MJX-45-TEX-N-2212",[171,4722],{"id":4723,"d":674},"MJX-45-TEX-I-1D466",[208,4725,4726],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,4727,4728,4733,4738,4742,4748,4754,4760,4766],{"dataMmlNode":215},[208,4729,4730],{"dataMmlNode":218},[220,4731],{"dataC":222,"xLinkHref":4732},"#MJX-45-TEX-N-28",[208,4734,4735],{"dataMmlNode":226,"transform":227},[220,4736],{"dataC":876,"xLinkHref":4737},"#MJX-45-TEX-I-210E",[208,4739,4740],{"dataMmlNode":218,"transform":886},[220,4741],{"dataC":222,"xLinkHref":4732},[208,4743,4745],{"dataMmlNode":226,"transform":4744},"translate(1354,0)",[220,4746],{"dataC":230,"xLinkHref":4747},"#MJX-45-TEX-I-1D465",[208,4749,4751],{"dataMmlNode":218,"transform":4750},"translate(1926,0)",[220,4752],{"dataC":265,"xLinkHref":4753},"#MJX-45-TEX-N-29",[208,4755,4757],{"dataMmlNode":218,"transform":4756},"translate(2537.2,0)",[220,4758],{"dataC":237,"xLinkHref":4759},"#MJX-45-TEX-N-2212",[208,4761,4763],{"dataMmlNode":226,"transform":4762},"translate(3537.4,0)",[220,4764],{"dataC":694,"xLinkHref":4765},"#MJX-45-TEX-I-1D466",[208,4767,4769],{"dataMmlNode":218,"transform":4768},"translate(4027.4,0)",[220,4770],{"dataC":265,"xLinkHref":4753}," to ",[151,4773,4775],{"className":4774,"jax":155},[154],[157,4776,4777,4797],{"style":159,"xmlns":160,"width":4702,"height":162,"role":163,"focusable":164,"viewBox":4703,"xmlnsXLink":166},[168,4778,4779,4782,4785,4788,4791,4794],{},[171,4780],{"id":4781,"d":174},"MJX-46-TEX-N-28",[171,4783],{"id":4784,"d":674},"MJX-46-TEX-I-1D466",[171,4786],{"id":4787,"d":182},"MJX-46-TEX-N-2212",[171,4789],{"id":4790,"d":858},"MJX-46-TEX-I-210E",[171,4792],{"id":4793,"d":178},"MJX-46-TEX-I-1D465",[171,4795],{"id":4796,"d":198},"MJX-46-TEX-N-29",[208,4798,4799],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,4800,4801,4806,4811,4817,4823,4828,4834,4840],{"dataMmlNode":215},[208,4802,4803],{"dataMmlNode":218},[220,4804],{"dataC":222,"xLinkHref":4805},"#MJX-46-TEX-N-28",[208,4807,4808],{"dataMmlNode":226,"transform":227},[220,4809],{"dataC":694,"xLinkHref":4810},"#MJX-46-TEX-I-1D466",[208,4812,4814],{"dataMmlNode":218,"transform":4813},"translate(1101.2,0)",[220,4815],{"dataC":237,"xLinkHref":4816},"#MJX-46-TEX-N-2212",[208,4818,4820],{"dataMmlNode":226,"transform":4819},"translate(2101.4,0)",[220,4821],{"dataC":876,"xLinkHref":4822},"#MJX-46-TEX-I-210E",[208,4824,4826],{"dataMmlNode":218,"transform":4825},"translate(2677.4,0)",[220,4827],{"dataC":222,"xLinkHref":4805},[208,4829,4831],{"dataMmlNode":226,"transform":4830},"translate(3066.4,0)",[220,4832],{"dataC":230,"xLinkHref":4833},"#MJX-46-TEX-I-1D465",[208,4835,4837],{"dataMmlNode":218,"transform":4836},"translate(3638.4,0)",[220,4838],{"dataC":265,"xLinkHref":4839},"#MJX-46-TEX-N-29",[208,4841,4842],{"dataMmlNode":218,"transform":4768},[220,4843],{"dataC":265,"xLinkHref":4839},[16,4845,4846],{},[20,4847,4848],{},"For Batch Gradient Descent (All Samples):",[151,4850,4852],{"className":4851,"jax":155,"display":1017},[154],[157,4853,4854,4913],{"style":1241,"xmlns":160,"width":4459,"height":4460,"role":163,"focusable":164,"viewBox":4461,"xmlnsXLink":166},[168,4855,4856,4859,4862,4865,4868,4871,4874,4877,4880,4883,4886,4889,4892,4895,4898,4901,4904,4907,4910],{},[171,4857],{"id":4858,"d":1043},"MJX-47-TEX-I-1D703",[171,4860],{"id":4861,"d":1732},"MJX-47-TEX-I-1D457",[171,4863],{"id":4864,"d":1736},"MJX-47-TEX-N-3A",[171,4866],{"id":4867,"d":1039},"MJX-47-TEX-N-3D",[171,4869],{"id":4870,"d":1051},"MJX-47-TEX-N-2B",[171,4872],{"id":4873,"d":1746},"MJX-47-TEX-I-1D6FC",[171,4875],{"id":4876,"d":1272},"MJX-47-TEX-LO-2211",[171,4878],{"id":4879,"d":190},"MJX-47-TEX-I-1D456",[171,4881],{"id":4882,"d":4233},"MJX-47-TEX-LO-5B",[171,4884],{"id":4885,"d":2384},"MJX-47-TEX-LO-28",[171,4887],{"id":4888,"d":674},"MJX-47-TEX-I-1D466",[171,4890],{"id":4891,"d":174},"MJX-47-TEX-N-28",[171,4893],{"id":4894,"d":198},"MJX-47-TEX-N-29",[171,4896],{"id":4897,"d":182},"MJX-47-TEX-N-2212",[171,4899],{"id":4900,"d":858},"MJX-47-TEX-I-210E",[171,4902],{"id":4903,"d":178},"MJX-47-TEX-I-1D465",[171,4905],{"id":4906,"d":2400},"MJX-47-TEX-LO-29",[171,4908],{"id":4909,"d":2872},"MJX-47-TEX-N-22C5",[171,4911],{"id":4912,"d":4258},"MJX-47-TEX-LO-5D",[208,4914,4915],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,4916,4917,4929,4941,4951,4956,4961,4973],{"dataMmlNode":215},[208,4918,4919,4924],{"dataMmlNode":1093},[208,4920,4921],{"dataMmlNode":226},[220,4922],{"dataC":1099,"xLinkHref":4923},"#MJX-47-TEX-I-1D703",[208,4925,4926],{"dataMmlNode":226,"transform":1104},[220,4927],{"dataC":1775,"xLinkHref":4928},"#MJX-47-TEX-I-1D457",[208,4930,4931,4936],{"dataMmlNode":218,"transform":1779},[208,4932,4933],{"dataMmlNode":980},[220,4934],{"dataC":1784,"xLinkHref":4935},"#MJX-47-TEX-N-3A",[208,4937,4938],{"dataMmlNode":980,"transform":1788},[220,4939],{"dataC":1089,"xLinkHref":4940},"#MJX-47-TEX-N-3D",[208,4942,4943,4947],{"dataMmlNode":1093,"transform":1794},[208,4944,4945],{"dataMmlNode":226},[220,4946],{"dataC":1099,"xLinkHref":4923},[208,4948,4949],{"dataMmlNode":226,"transform":1104},[220,4950],{"dataC":1775,"xLinkHref":4928},[208,4952,4953],{"dataMmlNode":218,"transform":1805},[220,4954],{"dataC":1114,"xLinkHref":4955},"#MJX-47-TEX-N-2B",[208,4957,4958],{"dataMmlNode":226,"transform":1811},[220,4959],{"dataC":1814,"xLinkHref":4960},"#MJX-47-TEX-I-1D6FC",[208,4962,4963,4968],{"dataMmlNode":1347,"transform":4568},[208,4964,4965],{"dataMmlNode":218},[220,4966],{"dataC":1353,"xLinkHref":4967},"#MJX-47-TEX-LO-2211",[208,4969,4970],{"dataMmlNode":226,"transform":1357},[220,4971],{"dataC":251,"xLinkHref":4972},"#MJX-47-TEX-I-1D456",[208,4974,4975,4980,5059,5064,5088],{"dataMmlNode":1821,"transform":4581},[208,4976,4977],{"dataMmlNode":218,"transform":2463},[220,4978],{"dataC":4334,"xLinkHref":4979},"#MJX-47-TEX-LO-5B",[208,4981,4982,4987,5010,5016,5022,5027,5049,5054],{"dataMmlNode":1821,"transform":4338},[208,4983,4984],{"dataMmlNode":218,"transform":2463},[220,4985],{"dataC":222,"xLinkHref":4986},"#MJX-47-TEX-LO-28",[208,4988,4989,4994],{"dataMmlNode":630,"transform":2469},[208,4990,4991],{"dataMmlNode":226},[220,4992],{"dataC":694,"xLinkHref":4993},"#MJX-47-TEX-I-1D466",[208,4995,4996,5001,5005],{"dataMmlNode":269,"transform":1421,"dataMjxTexclass":270},[208,4997,4998],{"dataMmlNode":218},[220,4999],{"dataC":222,"xLinkHref":5000},"#MJX-47-TEX-N-28",[208,5002,5003],{"dataMmlNode":226,"transform":227},[220,5004],{"dataC":251,"xLinkHref":4972},[208,5006,5007],{"dataMmlNode":218,"transform":651},[220,5008],{"dataC":265,"xLinkHref":5009},"#MJX-47-TEX-N-29",[208,5011,5013],{"dataMmlNode":218,"transform":5012},"translate(2186.3,0)",[220,5014],{"dataC":237,"xLinkHref":5015},"#MJX-47-TEX-N-2212",[208,5017,5019],{"dataMmlNode":226,"transform":5018},"translate(3186.5,0)",[220,5020],{"dataC":876,"xLinkHref":5021},"#MJX-47-TEX-I-210E",[208,5023,5025],{"dataMmlNode":218,"transform":5024},"translate(3762.5,0)",[220,5026],{"dataC":222,"xLinkHref":5000},[208,5028,5030,5035],{"dataMmlNode":630,"transform":5029},"translate(4151.5,0)",[208,5031,5032],{"dataMmlNode":226},[220,5033],{"dataC":230,"xLinkHref":5034},"#MJX-47-TEX-I-1D465",[208,5036,5037,5041,5045],{"dataMmlNode":269,"transform":1387,"dataMjxTexclass":270},[208,5038,5039],{"dataMmlNode":218},[220,5040],{"dataC":222,"xLinkHref":5000},[208,5042,5043],{"dataMmlNode":226,"transform":227},[220,5044],{"dataC":251,"xLinkHref":4972},[208,5046,5047],{"dataMmlNode":218,"transform":651},[220,5048],{"dataC":265,"xLinkHref":5009},[208,5050,5052],{"dataMmlNode":218,"transform":5051},"translate(5600.6,0)",[220,5053],{"dataC":265,"xLinkHref":5009},[208,5055,5056],{"dataMmlNode":218,"transform":2535},[220,5057],{"dataC":265,"xLinkHref":5058},"#MJX-47-TEX-LO-29",[208,5060,5061],{"dataMmlNode":218,"transform":4411},[220,5062],{"dataC":3035,"xLinkHref":5063},"#MJX-47-TEX-N-22C5",[208,5065,5066,5070,5084],{"dataMmlNode":2138,"transform":4417},[208,5067,5068],{"dataMmlNode":226},[220,5069],{"dataC":230,"xLinkHref":5034},[208,5071,5072,5076,5080],{"dataMmlNode":269,"transform":2146,"dataMjxTexclass":270},[208,5073,5074],{"dataMmlNode":218},[220,5075],{"dataC":222,"xLinkHref":5000},[208,5077,5078],{"dataMmlNode":226,"transform":227},[220,5079],{"dataC":251,"xLinkHref":4972},[208,5081,5082],{"dataMmlNode":218,"transform":651},[220,5083],{"dataC":265,"xLinkHref":5009},[208,5085,5086],{"dataMmlNode":226,"transform":2161},[220,5087],{"dataC":1775,"xLinkHref":4928},[208,5089,5090],{"dataMmlNode":218,"transform":4442},[220,5091],{"dataC":4445,"xLinkHref":5092},"#MJX-47-TEX-LO-5D",[16,5094,5095,5098],{},[20,5096,5097],{},"For Stochastic Gradient Descent (1 Sample at a time):","\nWe simply drop the summation, giving us the exact LMS update rule:",[151,5100,5102],{"className":5101,"jax":155,"display":1017},[154],[157,5103,5108,5158],{"style":5104,"xmlns":160,"width":5105,"height":5106,"role":163,"focusable":164,"viewBox":5107,"xmlnsXLink":166},"vertical-align: -1.469ex;","31.868ex","4.07ex","0 -1149.5 14085.5 1799",[168,5109,5110,5113,5116,5119,5122,5125,5128,5131,5134,5137,5140,5143,5146,5149,5152,5155],{},[171,5111],{"id":5112,"d":1043},"MJX-48-TEX-I-1D703",[171,5114],{"id":5115,"d":1732},"MJX-48-TEX-I-1D457",[171,5117],{"id":5118,"d":1736},"MJX-48-TEX-N-3A",[171,5120],{"id":5121,"d":1039},"MJX-48-TEX-N-3D",[171,5123],{"id":5124,"d":1051},"MJX-48-TEX-N-2B",[171,5126],{"id":5127,"d":1746},"MJX-48-TEX-I-1D6FC",[171,5129],{"id":5130,"d":2384},"MJX-48-TEX-LO-28",[171,5132],{"id":5133,"d":674},"MJX-48-TEX-I-1D466",[171,5135],{"id":5136,"d":174},"MJX-48-TEX-N-28",[171,5138],{"id":5139,"d":190},"MJX-48-TEX-I-1D456",[171,5141],{"id":5142,"d":198},"MJX-48-TEX-N-29",[171,5144],{"id":5145,"d":182},"MJX-48-TEX-N-2212",[171,5147],{"id":5148,"d":858},"MJX-48-TEX-I-210E",[171,5150],{"id":5151,"d":178},"MJX-48-TEX-I-1D465",[171,5153],{"id":5154,"d":2400},"MJX-48-TEX-LO-29",[171,5156],{"id":5157,"d":2872},"MJX-48-TEX-N-22C5",[208,5159,5160],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,5161,5162,5174,5186,5196,5201,5206,5281,5287],{"dataMmlNode":215},[208,5163,5164,5169],{"dataMmlNode":1093},[208,5165,5166],{"dataMmlNode":226},[220,5167],{"dataC":1099,"xLinkHref":5168},"#MJX-48-TEX-I-1D703",[208,5170,5171],{"dataMmlNode":226,"transform":1104},[220,5172],{"dataC":1775,"xLinkHref":5173},"#MJX-48-TEX-I-1D457",[208,5175,5176,5181],{"dataMmlNode":218,"transform":1779},[208,5177,5178],{"dataMmlNode":980},[220,5179],{"dataC":1784,"xLinkHref":5180},"#MJX-48-TEX-N-3A",[208,5182,5183],{"dataMmlNode":980,"transform":1788},[220,5184],{"dataC":1089,"xLinkHref":5185},"#MJX-48-TEX-N-3D",[208,5187,5188,5192],{"dataMmlNode":1093,"transform":1794},[208,5189,5190],{"dataMmlNode":226},[220,5191],{"dataC":1099,"xLinkHref":5168},[208,5193,5194],{"dataMmlNode":226,"transform":1104},[220,5195],{"dataC":1775,"xLinkHref":5173},[208,5197,5198],{"dataMmlNode":218,"transform":1805},[220,5199],{"dataC":1114,"xLinkHref":5200},"#MJX-48-TEX-N-2B",[208,5202,5203],{"dataMmlNode":226,"transform":1811},[220,5204],{"dataC":1814,"xLinkHref":5205},"#MJX-48-TEX-I-1D6FC",[208,5207,5208,5213,5237,5242,5247,5251,5272,5276],{"dataMmlNode":1821,"transform":4568},[208,5209,5210],{"dataMmlNode":218,"transform":2463},[220,5211],{"dataC":222,"xLinkHref":5212},"#MJX-48-TEX-LO-28",[208,5214,5215,5220],{"dataMmlNode":630,"transform":2469},[208,5216,5217],{"dataMmlNode":226},[220,5218],{"dataC":694,"xLinkHref":5219},"#MJX-48-TEX-I-1D466",[208,5221,5222,5227,5232],{"dataMmlNode":269,"transform":1421,"dataMjxTexclass":270},[208,5223,5224],{"dataMmlNode":218},[220,5225],{"dataC":222,"xLinkHref":5226},"#MJX-48-TEX-N-28",[208,5228,5229],{"dataMmlNode":226,"transform":227},[220,5230],{"dataC":251,"xLinkHref":5231},"#MJX-48-TEX-I-1D456",[208,5233,5234],{"dataMmlNode":218,"transform":651},[220,5235],{"dataC":265,"xLinkHref":5236},"#MJX-48-TEX-N-29",[208,5238,5239],{"dataMmlNode":218,"transform":5012},[220,5240],{"dataC":237,"xLinkHref":5241},"#MJX-48-TEX-N-2212",[208,5243,5244],{"dataMmlNode":226,"transform":5018},[220,5245],{"dataC":876,"xLinkHref":5246},"#MJX-48-TEX-I-210E",[208,5248,5249],{"dataMmlNode":218,"transform":5024},[220,5250],{"dataC":222,"xLinkHref":5226},[208,5252,5253,5258],{"dataMmlNode":630,"transform":5029},[208,5254,5255],{"dataMmlNode":226},[220,5256],{"dataC":230,"xLinkHref":5257},"#MJX-48-TEX-I-1D465",[208,5259,5260,5264,5268],{"dataMmlNode":269,"transform":1387,"dataMjxTexclass":270},[208,5261,5262],{"dataMmlNode":218},[220,5263],{"dataC":222,"xLinkHref":5226},[208,5265,5266],{"dataMmlNode":226,"transform":227},[220,5267],{"dataC":251,"xLinkHref":5231},[208,5269,5270],{"dataMmlNode":218,"transform":651},[220,5271],{"dataC":265,"xLinkHref":5236},[208,5273,5274],{"dataMmlNode":218,"transform":5051},[220,5275],{"dataC":265,"xLinkHref":5236},[208,5277,5278],{"dataMmlNode":218,"transform":2535},[220,5279],{"dataC":265,"xLinkHref":5280},"#MJX-48-TEX-LO-29",[208,5282,5284],{"dataMmlNode":218,"transform":5283},"translate(12136.2,0)",[220,5285],{"dataC":3035,"xLinkHref":5286},"#MJX-48-TEX-N-22C5",[208,5288,5290,5294,5308],{"dataMmlNode":2138,"transform":5289},"translate(12636.4,0)",[208,5291,5292],{"dataMmlNode":226},[220,5293],{"dataC":230,"xLinkHref":5257},[208,5295,5296,5300,5304],{"dataMmlNode":269,"transform":2146,"dataMjxTexclass":270},[208,5297,5298],{"dataMmlNode":218},[220,5299],{"dataC":222,"xLinkHref":5226},[208,5301,5302],{"dataMmlNode":226,"transform":227},[220,5303],{"dataC":251,"xLinkHref":5231},[208,5305,5306],{"dataMmlNode":218,"transform":651},[220,5307],{"dataC":265,"xLinkHref":5236},[208,5309,5310],{"dataMmlNode":226,"transform":2161},[220,5311],{"dataC":1775,"xLinkHref":5173},[28,5313,5315],{"id":5314},"batch-vs-stochastic-gradient-descent","Batch vs. Stochastic Gradient Descent",[16,5317,5318,5321,5322,5325],{},[20,5319,5320],{},"Batch Gradient Descent"," sums the gradients over ",[87,5323,5324],{},"all"," training examples before making a single update. It's precise but can be slow for large datasets.",[16,5327,5328],{},[163,5329],{"alt":5330,"src":5331},"0.84","https:\u002F\u002Fraw.githubusercontent.com\u002FChinHongTan\u002Fblog\u002Fmain\u002Fpublic\u002Fimages\u002Fuploads\u002F1777294390662-1775987318720-Screenshot_2026-04-12_at_5.webp",[16,5333,5334,5337,5338,5357,5358,5361],{},[20,5335,5336],{},"Stochastic (Incremental) Gradient Descent"," updates ",[151,5339,5341],{"className":5340,"jax":155},[154],[157,5342,5343,5348],{"style":1178,"xmlns":160,"width":1179,"height":1180,"role":163,"focusable":164,"viewBox":1181,"xmlnsXLink":166},[168,5344,5345],{},[171,5346],{"id":5347,"d":1043},"MJX-49-TEX-I-1D703",[208,5349,5350],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,5351,5352],{"dataMmlNode":215},[208,5353,5354],{"dataMmlNode":226},[220,5355],{"dataC":1099,"xLinkHref":5356},"#MJX-49-TEX-I-1D703"," after ",[87,5359,5360],{},"each"," training example. It's faster and often works well in practice, though the path to convergence is noisier.",[16,5363,5364],{},[163,5365],{"alt":5366,"src":5367},"0.70","https:\u002F\u002Fraw.githubusercontent.com\u002FChinHongTan\u002Fblog\u002Fmain\u002Fpublic\u002Fimages\u002Fuploads\u002F1777294405734-1775987315179-Screenshot_2026-04-12_at_5.webp",[28,5369,5371],{"id":5370},"underfitting-vs-overfitting","Underfitting vs. Overfitting",[16,5373,5374],{},"This is one of the most important concepts in ML. Consider fitting different models to our housing data:",[520,5376,5377,5387,5420],{},[523,5378,5379,5380,5383,5384,1542],{},"A ",[20,5381,5382],{},"straight line"," (linear fit) may be too simple — it doesn't capture the curve in the data. This is ",[20,5385,5386],{},"underfitting",[523,5388,5379,5389,5392,5393,5419],{},[20,5390,5391],{},"quadratic"," (adding ",[151,5394,5396],{"className":5395,"jax":155},[154],[157,5397,5398,5403],{"style":956,"xmlns":160,"width":957,"height":958,"role":163,"focusable":164,"viewBox":959,"xmlnsXLink":166},[168,5399,5400],{},[171,5401],{"id":5402,"d":178},"MJX-50-TEX-I-1D465",[208,5404,5405],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,5406,5407,5412],{"dataMmlNode":215},[208,5408,5409],{"dataMmlNode":226},[220,5410],{"dataC":230,"xLinkHref":5411},"#MJX-50-TEX-I-1D465",[208,5413,5414],{"dataMmlNode":269,"dataMjxTexclass":270,"transform":976},[208,5415,5416],{"dataMmlNode":218},[980,5417,5418],{"dataVariant":982,"transform":212,"font-size":983,"font-family":984},"²",") fits better, capturing more structure.",[523,5421,5379,5422,5425,5426,1542],{},[20,5423,5424],{},"5th-order polynomial"," passes through every data point perfectly, but it would make terrible predictions on new data. This is ",[20,5427,5428],{},"overfitting",[16,5430,5431],{},[163,5432],{"alt":5433,"src":5434},"0.90","https:\u002F\u002Fraw.githubusercontent.com\u002FChinHongTan\u002Fblog\u002Fmain\u002Fpublic\u002Fimages\u002Fuploads\u002F1777294461971-1775987112930-Screenshot_2026-04-12_at_5.webp",[16,5436,5437,5438,5441],{},"The key insight: a model that memorizes training data is not necessarily a good model. We want one that ",[20,5439,5440],{},"generalizes"," to unseen data.",[28,5443,5445],{"id":5444},"locally-weighted-linear-regression","Locally Weighted Linear Regression",[16,5447,5448,5449,5452],{},"Standard linear regression fits one global line to all data. ",[20,5450,5451],{},"Locally Weighted Linear Regression (LWLR)"," takes a different approach: for each prediction, it gives more importance to nearby training points and less to distant ones.",[16,5454,5455],{},"The weight for each training example is typically a Gaussian function:",[151,5457,5459],{"className":5458,"jax":155,"display":1017},[154],[157,5460,5463,5516],{"style":2826,"xmlns":160,"width":5461,"height":2828,"role":163,"focusable":164,"viewBox":5462,"xmlnsXLink":166},"26.881ex","0 -1749.5 11881.4 2999",[168,5464,5465,5469,5472,5475,5478,5481,5485,5489,5493,5497,5500,5503,5506,5509,5513],{},[171,5466],{"id":5467,"d":5468},"MJX-51-TEX-I-1D464","M580 385Q580 406 599 424T641 443Q659 443 674 425T690 368Q690 339 671 253Q656 197 644 161T609 80T554 12T482 -11Q438 -11 404 5T355 48Q354 47 352 44Q311 -11 252 -11Q226 -11 202 -5T155 14T118 53T104 116Q104 170 138 262T173 379Q173 380 173 381Q173 390 173 393T169 400T158 404H154Q131 404 112 385T82 344T65 302T57 280Q55 278 41 278H27Q21 284 21 287Q21 293 29 315T52 366T96 418T161 441Q204 441 227 416T250 358Q250 340 217 250T184 111Q184 65 205 46T258 26Q301 26 334 87L339 96V119Q339 122 339 128T340 136T341 143T342 152T345 165T348 182T354 206T362 238T373 281Q402 395 406 404Q419 431 449 431Q468 431 475 421T483 402Q483 389 454 274T422 142Q420 131 420 107V100Q420 85 423 71T442 42T487 26Q558 26 600 148Q609 171 620 213T632 273Q632 306 619 325T593 357T580 385Z",[171,5470],{"id":5471,"d":174},"MJX-51-TEX-N-28",[171,5473],{"id":5474,"d":190},"MJX-51-TEX-I-1D456",[171,5476],{"id":5477,"d":198},"MJX-51-TEX-N-29",[171,5479],{"id":5480,"d":1039},"MJX-51-TEX-N-3D",[171,5482],{"id":5483,"d":5484},"MJX-51-TEX-N-65","M28 218Q28 273 48 318T98 391T163 433T229 448Q282 448 320 430T378 380T406 316T415 245Q415 238 408 231H126V216Q126 68 226 36Q246 30 270 30Q312 30 342 62Q359 79 369 104L379 128Q382 131 395 131H398Q415 131 415 121Q415 117 412 108Q393 53 349 21T250 -11Q155 -11 92 58T28 218ZM333 275Q322 403 238 411H236Q228 411 220 410T195 402T166 381T143 340T127 274V267H333V275Z",[171,5486],{"id":5487,"d":5488},"MJX-51-TEX-N-78","M201 0Q189 3 102 3Q26 3 17 0H11V46H25Q48 47 67 52T96 61T121 78T139 96T160 122T180 150L226 210L168 288Q159 301 149 315T133 336T122 351T113 363T107 370T100 376T94 379T88 381T80 383Q74 383 44 385H16V431H23Q59 429 126 429Q219 429 229 431H237V385Q201 381 201 369Q201 367 211 353T239 315T268 274L272 270L297 304Q329 345 329 358Q329 364 327 369T322 376T317 380T310 384L307 385H302V431H309Q324 428 408 428Q487 428 493 431H499V385H492Q443 385 411 368Q394 360 377 341T312 257L296 236L358 151Q424 61 429 57T446 50Q464 46 499 46H516V0H510H502Q494 1 482 1T457 2T432 2T414 3Q403 3 377 3T327 1L304 0H295V46H298Q309 46 320 51T331 63Q331 65 291 120L250 175Q249 174 219 133T185 88Q181 83 181 74Q181 63 188 55T206 46Q208 46 208 23V0H201Z",[171,5490],{"id":5491,"d":5492},"MJX-51-TEX-N-70","M36 -148H50Q89 -148 97 -134V-126Q97 -119 97 -107T97 -77T98 -38T98 6T98 55T98 106Q98 140 98 177T98 243T98 296T97 335T97 351Q94 370 83 376T38 385H20V408Q20 431 22 431L32 432Q42 433 61 434T98 436Q115 437 135 438T165 441T176 442H179V416L180 390L188 397Q247 441 326 441Q407 441 464 377T522 216Q522 115 457 52T310 -11Q242 -11 190 33L182 40V-45V-101Q182 -128 184 -134T195 -145Q216 -148 244 -148H260V-194H252L228 -193Q205 -192 178 -192T140 -191Q37 -191 28 -194H20V-148H36ZM424 218Q424 292 390 347T305 402Q234 402 182 337V98Q222 26 294 26Q345 26 384 80T424 218Z",[171,5494],{"id":5495,"d":5496},"MJX-51-TEX-N-2061","",[171,5498],{"id":5499,"d":2862},"MJX-51-TEX-S4-28",[171,5501],{"id":5502,"d":182},"MJX-51-TEX-N-2212",[171,5504],{"id":5505,"d":178},"MJX-51-TEX-I-1D465",[171,5507],{"id":5508,"d":1059},"MJX-51-TEX-N-32",[171,5510],{"id":5511,"d":5512},"MJX-51-TEX-I-1D70F","M39 284Q18 284 18 294Q18 301 45 338T99 398Q134 425 164 429Q170 431 332 431Q492 431 497 429Q517 424 517 402Q517 388 508 376T485 360Q479 358 389 358T299 356Q298 355 283 274T251 109T233 20Q228 5 215 -4T186 -13Q153 -13 153 20V30L203 192Q214 228 227 272T248 336L254 357Q254 358 208 358Q206 358 197 358T183 359Q105 359 61 295Q56 287 53 286T39 284Z",[171,5514],{"id":5515,"d":2876},"MJX-51-TEX-S4-29",[208,5517,5518],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,5519,5520,5546,5552,5569,5576],{"dataMmlNode":215},[208,5521,5522,5528],{"dataMmlNode":630},[208,5523,5524],{"dataMmlNode":226},[220,5525],{"dataC":5526,"xLinkHref":5527},"1D464","#MJX-51-TEX-I-1D464",[208,5529,5531,5536,5541],{"dataMmlNode":269,"transform":5530,"dataMjxTexclass":270},"translate(749,413) scale(0.707)",[208,5532,5533],{"dataMmlNode":218},[220,5534],{"dataC":222,"xLinkHref":5535},"#MJX-51-TEX-N-28",[208,5537,5538],{"dataMmlNode":226,"transform":227},[220,5539],{"dataC":251,"xLinkHref":5540},"#MJX-51-TEX-I-1D456",[208,5542,5543],{"dataMmlNode":218,"transform":651},[220,5544],{"dataC":265,"xLinkHref":5545},"#MJX-51-TEX-N-29",[208,5547,5549],{"dataMmlNode":218,"transform":5548},"translate(1870.9,0)",[220,5550],{"dataC":1089,"xLinkHref":5551},"#MJX-51-TEX-N-3D",[208,5553,5555,5559,5564],{"dataMmlNode":226,"transform":5554},"translate(2926.6,0)",[220,5556],{"dataC":5557,"xLinkHref":5558},"65","#MJX-51-TEX-N-65",[220,5560],{"dataC":5561,"xLinkHref":5562,"transform":5563},"78","#MJX-51-TEX-N-78","translate(444,0)",[220,5565],{"dataC":5566,"xLinkHref":5567,"transform":5568},"70","#MJX-51-TEX-N-70","translate(972,0)",[208,5570,5572],{"dataMmlNode":218,"transform":5571},"translate(4454.6,0)",[220,5573],{"dataC":5574,"xLinkHref":5575},"2061","#MJX-51-TEX-N-2061",[208,5577,5579,5584,5589,5665],{"dataMmlNode":1821,"transform":5578},"translate(4621.3,0)",[208,5580,5581],{"dataMmlNode":218,"transform":2463},[220,5582],{"dataC":222,"xLinkHref":5583},"#MJX-51-TEX-S4-28",[208,5585,5586],{"dataMmlNode":218,"transform":2958},[220,5587],{"dataC":237,"xLinkHref":5588},"#MJX-51-TEX-N-2212",[208,5590,5592,5642,5662],{"dataMmlNode":1324,"transform":5591},"translate(1570,0)",[208,5593,5594,5598,5619,5624,5629],{"dataMmlNode":1821,"transform":1822},[208,5595,5596],{"dataMmlNode":218},[220,5597],{"dataC":222,"xLinkHref":5535},[208,5599,5600,5605],{"dataMmlNode":630,"transform":227},[208,5601,5602],{"dataMmlNode":226},[220,5603],{"dataC":230,"xLinkHref":5604},"#MJX-51-TEX-I-1D465",[208,5606,5607,5611,5615],{"dataMmlNode":269,"transform":638,"dataMjxTexclass":270},[208,5608,5609],{"dataMmlNode":218},[220,5610],{"dataC":222,"xLinkHref":5535},[208,5612,5613],{"dataMmlNode":226,"transform":227},[220,5614],{"dataC":251,"xLinkHref":5540},[208,5616,5617],{"dataMmlNode":218,"transform":651},[220,5618],{"dataC":265,"xLinkHref":5545},[208,5620,5622],{"dataMmlNode":218,"transform":5621},"translate(2060.3,0)",[220,5623],{"dataC":237,"xLinkHref":5588},[208,5625,5627],{"dataMmlNode":226,"transform":5626},"translate(3060.5,0)",[220,5628],{"dataC":230,"xLinkHref":5604},[208,5630,5632,5636],{"dataMmlNode":630,"transform":5631},"translate(3632.5,0)",[208,5633,5634],{"dataMmlNode":218},[220,5635],{"dataC":265,"xLinkHref":5545},[208,5637,5639],{"dataMmlNode":1103,"transform":5638},"translate(422,363) scale(0.707)",[220,5640],{"dataC":1157,"xLinkHref":5641},"#MJX-51-TEX-N-32",[208,5643,5645,5649],{"dataMmlNode":1821,"transform":5644},"translate(1695.3,-719.9)",[208,5646,5647],{"dataMmlNode":1103},[220,5648],{"dataC":1157,"xLinkHref":5641},[208,5650,5651,5657],{"dataMmlNode":630,"transform":3275},[208,5652,5653],{"dataMmlNode":226},[220,5654],{"dataC":5655,"xLinkHref":5656},"1D70F","#MJX-51-TEX-I-1D70F",[208,5658,5660],{"dataMmlNode":1103,"transform":5659},"translate(604,289) scale(0.707)",[220,5661],{"dataC":1157,"xLinkHref":5641},[1339,5663],{"width":5664,"height":1342,"x":1343,"y":1344},4658.1,[208,5666,5668],{"dataMmlNode":218,"transform":5667},"translate(6468.1,0) translate(0 -0.5)",[220,5669],{"dataC":265,"xLinkHref":5670},"#MJX-51-TEX-S4-29",[16,5672,5673,5674,5697,5698,5701],{},"The parameter ",[151,5675,5677],{"className":5676,"jax":155},[154],[157,5678,5683,5688],{"style":5679,"xmlns":160,"width":5680,"height":5681,"role":163,"focusable":164,"viewBox":5682,"xmlnsXLink":166},"vertical-align: -0.029ex;","1.17ex","1.005ex","0 -431 517 444",[168,5684,5685],{},[171,5686],{"id":5687,"d":5512},"MJX-52-TEX-I-1D70F",[208,5689,5690],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,5691,5692],{"dataMmlNode":215},[208,5693,5694],{"dataMmlNode":226},[220,5695],{"dataC":5655,"xLinkHref":5696},"#MJX-52-TEX-I-1D70F"," (tau) controls how quickly the weights decay with distance. LWLR is a ",[20,5699,5700],{},"non-parametric"," method — it doesn't commit to a fixed set of parameters but re-fits for each query point.",[69,5703,5705,5708],{"className":5704},[72,73],[75,5706,5707],{},"The Intuition behind Locally Weighted Linear Regression (LWLR)",[79,5709,5711,5718,5750,5803,5807,5810,5995,5998,6645,6649,6774],{"className":5710},[82],[16,5712,5713,5714,5717],{},"In standard linear regression, when you fit a line, ",[20,5715,5716],{},"every single training example has an equal vote",". A house that is 5,000 sq ft has the exact same influence on the line as a house that is 1,000 sq ft.",[16,5719,5720,5721,5724,5725,5746,5747],{},"LWLR changes this. If you want to predict the price of a house that is ",[20,5722,5723],{},"1,500 sq ft"," (your query point, ",[151,5726,5728],{"className":5727,"jax":155},[154],[157,5729,5732,5737],{"style":606,"xmlns":160,"width":5730,"height":825,"role":163,"focusable":164,"viewBox":5731,"xmlnsXLink":166},"1.294ex","0 -442 572 453",[168,5733,5734],{},[171,5735],{"id":5736,"d":178},"MJX-53-TEX-I-1D465",[208,5738,5739],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,5740,5741],{"dataMmlNode":215},[208,5742,5743],{"dataMmlNode":226},[220,5744],{"dataC":230,"xLinkHref":5745},"#MJX-53-TEX-I-1D465","), LWLR says: ",[87,5748,5749],{},"\"I only really care about training examples that are similar in size to 1,500 sq ft. The 5,000 sq ft houses shouldn't influence my prediction right now.\"",[16,5751,5752,5753,1542],{},"To do this, we multiply the error of each training example by a weight, ",[151,5754,5756],{"className":5755,"jax":155},[154],[157,5757,5760,5774],{"style":606,"xmlns":160,"width":5758,"height":608,"role":163,"focusable":164,"viewBox":5759,"xmlnsXLink":166},"3.604ex","0 -893.3 1593.1 904.3",[168,5761,5762,5765,5768,5771],{},[171,5763],{"id":5764,"d":5468},"MJX-54-TEX-I-1D464",[171,5766],{"id":5767,"d":174},"MJX-54-TEX-N-28",[171,5769],{"id":5770,"d":190},"MJX-54-TEX-I-1D456",[171,5772],{"id":5773,"d":198},"MJX-54-TEX-N-29",[208,5775,5776],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,5777,5778],{"dataMmlNode":215},[208,5779,5780,5785],{"dataMmlNode":630},[208,5781,5782],{"dataMmlNode":226},[220,5783],{"dataC":5526,"xLinkHref":5784},"#MJX-54-TEX-I-1D464",[208,5786,5788,5793,5798],{"dataMmlNode":269,"transform":5787,"dataMjxTexclass":270},"translate(749,363) scale(0.707)",[208,5789,5790],{"dataMmlNode":218},[220,5791],{"dataC":222,"xLinkHref":5792},"#MJX-54-TEX-N-28",[208,5794,5795],{"dataMmlNode":226,"transform":227},[220,5796],{"dataC":251,"xLinkHref":5797},"#MJX-54-TEX-I-1D456",[208,5799,5800],{"dataMmlNode":218,"transform":651},[220,5801],{"dataC":265,"xLinkHref":5802},"#MJX-54-TEX-N-29",[28,5804,5806],{"id":5805},"deconstructing-the-equation","Deconstructing the Equation",[16,5808,5809],{},"The weight formula is a Gaussian (bell-shaped) curve:",[151,5811,5813],{"className":5812,"jax":155,"display":1017},[154],[157,5814,5815,5862],{"style":2826,"xmlns":160,"width":5461,"height":2828,"role":163,"focusable":164,"viewBox":5462,"xmlnsXLink":166},[168,5816,5817,5820,5823,5826,5829,5832,5835,5838,5841,5844,5847,5850,5853,5856,5859],{},[171,5818],{"id":5819,"d":5468},"MJX-55-TEX-I-1D464",[171,5821],{"id":5822,"d":174},"MJX-55-TEX-N-28",[171,5824],{"id":5825,"d":190},"MJX-55-TEX-I-1D456",[171,5827],{"id":5828,"d":198},"MJX-55-TEX-N-29",[171,5830],{"id":5831,"d":1039},"MJX-55-TEX-N-3D",[171,5833],{"id":5834,"d":5484},"MJX-55-TEX-N-65",[171,5836],{"id":5837,"d":5488},"MJX-55-TEX-N-78",[171,5839],{"id":5840,"d":5492},"MJX-55-TEX-N-70",[171,5842],{"id":5843,"d":5496},"MJX-55-TEX-N-2061",[171,5845],{"id":5846,"d":2862},"MJX-55-TEX-S4-28",[171,5848],{"id":5849,"d":182},"MJX-55-TEX-N-2212",[171,5851],{"id":5852,"d":178},"MJX-55-TEX-I-1D465",[171,5854],{"id":5855,"d":1059},"MJX-55-TEX-N-32",[171,5857],{"id":5858,"d":5512},"MJX-55-TEX-I-1D70F",[171,5860],{"id":5861,"d":2876},"MJX-55-TEX-S4-29",[208,5863,5864],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,5865,5866,5890,5895,5906,5911],{"dataMmlNode":215},[208,5867,5868,5873],{"dataMmlNode":630},[208,5869,5870],{"dataMmlNode":226},[220,5871],{"dataC":5526,"xLinkHref":5872},"#MJX-55-TEX-I-1D464",[208,5874,5875,5880,5885],{"dataMmlNode":269,"transform":5530,"dataMjxTexclass":270},[208,5876,5877],{"dataMmlNode":218},[220,5878],{"dataC":222,"xLinkHref":5879},"#MJX-55-TEX-N-28",[208,5881,5882],{"dataMmlNode":226,"transform":227},[220,5883],{"dataC":251,"xLinkHref":5884},"#MJX-55-TEX-I-1D456",[208,5886,5887],{"dataMmlNode":218,"transform":651},[220,5888],{"dataC":265,"xLinkHref":5889},"#MJX-55-TEX-N-29",[208,5891,5892],{"dataMmlNode":218,"transform":5548},[220,5893],{"dataC":1089,"xLinkHref":5894},"#MJX-55-TEX-N-3D",[208,5896,5897,5900,5903],{"dataMmlNode":226,"transform":5554},[220,5898],{"dataC":5557,"xLinkHref":5899},"#MJX-55-TEX-N-65",[220,5901],{"dataC":5561,"xLinkHref":5902,"transform":5563},"#MJX-55-TEX-N-78",[220,5904],{"dataC":5566,"xLinkHref":5905,"transform":5568},"#MJX-55-TEX-N-70",[208,5907,5908],{"dataMmlNode":218,"transform":5571},[220,5909],{"dataC":5574,"xLinkHref":5910},"#MJX-55-TEX-N-2061",[208,5912,5913,5918,5923,5990],{"dataMmlNode":1821,"transform":5578},[208,5914,5915],{"dataMmlNode":218,"transform":2463},[220,5916],{"dataC":222,"xLinkHref":5917},"#MJX-55-TEX-S4-28",[208,5919,5920],{"dataMmlNode":218,"transform":2958},[220,5921],{"dataC":237,"xLinkHref":5922},"#MJX-55-TEX-N-2212",[208,5924,5925,5971,5988],{"dataMmlNode":1324,"transform":5591},[208,5926,5927,5931,5952,5956,5960],{"dataMmlNode":1821,"transform":1822},[208,5928,5929],{"dataMmlNode":218},[220,5930],{"dataC":222,"xLinkHref":5879},[208,5932,5933,5938],{"dataMmlNode":630,"transform":227},[208,5934,5935],{"dataMmlNode":226},[220,5936],{"dataC":230,"xLinkHref":5937},"#MJX-55-TEX-I-1D465",[208,5939,5940,5944,5948],{"dataMmlNode":269,"transform":638,"dataMjxTexclass":270},[208,5941,5942],{"dataMmlNode":218},[220,5943],{"dataC":222,"xLinkHref":5879},[208,5945,5946],{"dataMmlNode":226,"transform":227},[220,5947],{"dataC":251,"xLinkHref":5884},[208,5949,5950],{"dataMmlNode":218,"transform":651},[220,5951],{"dataC":265,"xLinkHref":5889},[208,5953,5954],{"dataMmlNode":218,"transform":5621},[220,5955],{"dataC":237,"xLinkHref":5922},[208,5957,5958],{"dataMmlNode":226,"transform":5626},[220,5959],{"dataC":230,"xLinkHref":5937},[208,5961,5962,5966],{"dataMmlNode":630,"transform":5631},[208,5963,5964],{"dataMmlNode":218},[220,5965],{"dataC":265,"xLinkHref":5889},[208,5967,5968],{"dataMmlNode":1103,"transform":5638},[220,5969],{"dataC":1157,"xLinkHref":5970},"#MJX-55-TEX-N-32",[208,5972,5973,5977],{"dataMmlNode":1821,"transform":5644},[208,5974,5975],{"dataMmlNode":1103},[220,5976],{"dataC":1157,"xLinkHref":5970},[208,5978,5979,5984],{"dataMmlNode":630,"transform":3275},[208,5980,5981],{"dataMmlNode":226},[220,5982],{"dataC":5655,"xLinkHref":5983},"#MJX-55-TEX-I-1D70F",[208,5985,5986],{"dataMmlNode":1103,"transform":5659},[220,5987],{"dataC":1157,"xLinkHref":5970},[1339,5989],{"width":5664,"height":1342,"x":1343,"y":1344},[208,5991,5992],{"dataMmlNode":218,"transform":5667},[220,5993],{"dataC":265,"xLinkHref":5994},"#MJX-55-TEX-S4-29",[16,5996,5997],{},"Here is what each piece is doing:",[1575,5999,6000,6168,6484],{},[523,6001,6002,1498,6005,6084,6087,6088,6135,6136,1542,6155],{},[20,6003,6004],{},"The Distance:",[151,6006,6008],{"className":6007,"jax":155},[154],[157,6009,6012,6032],{"style":159,"xmlns":160,"width":6010,"height":725,"role":163,"focusable":164,"viewBox":6011,"xmlnsXLink":166},"10.086ex","0 -893.3 4458.1 1143.3",[168,6013,6014,6017,6020,6023,6026,6029],{},[171,6015],{"id":6016,"d":174},"MJX-56-TEX-N-28",[171,6018],{"id":6019,"d":178},"MJX-56-TEX-I-1D465",[171,6021],{"id":6022,"d":190},"MJX-56-TEX-I-1D456",[171,6024],{"id":6025,"d":198},"MJX-56-TEX-N-29",[171,6027],{"id":6028,"d":182},"MJX-56-TEX-N-2212",[171,6030],{"id":6031,"d":1059},"MJX-56-TEX-N-32",[208,6033,6034],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,6035,6036,6041,6064,6069,6073],{"dataMmlNode":215},[208,6037,6038],{"dataMmlNode":218},[220,6039],{"dataC":222,"xLinkHref":6040},"#MJX-56-TEX-N-28",[208,6042,6043,6048],{"dataMmlNode":630,"transform":227},[208,6044,6045],{"dataMmlNode":226},[220,6046],{"dataC":230,"xLinkHref":6047},"#MJX-56-TEX-I-1D465",[208,6049,6050,6054,6059],{"dataMmlNode":269,"transform":638,"dataMjxTexclass":270},[208,6051,6052],{"dataMmlNode":218},[220,6053],{"dataC":222,"xLinkHref":6040},[208,6055,6056],{"dataMmlNode":226,"transform":227},[220,6057],{"dataC":251,"xLinkHref":6058},"#MJX-56-TEX-I-1D456",[208,6060,6061],{"dataMmlNode":218,"transform":651},[220,6062],{"dataC":265,"xLinkHref":6063},"#MJX-56-TEX-N-29",[208,6065,6066],{"dataMmlNode":218,"transform":5621},[220,6067],{"dataC":237,"xLinkHref":6068},"#MJX-56-TEX-N-2212",[208,6070,6071],{"dataMmlNode":226,"transform":5626},[220,6072],{"dataC":230,"xLinkHref":6047},[208,6074,6075,6079],{"dataMmlNode":630,"transform":5631},[208,6076,6077],{"dataMmlNode":218},[220,6078],{"dataC":265,"xLinkHref":6063},[208,6080,6081],{"dataMmlNode":1103,"transform":5638},[220,6082],{"dataC":1157,"xLinkHref":6083},"#MJX-56-TEX-N-32",[6085,6086],"br",{},"This is the squared distance between a training example ",[151,6089,6091],{"className":6090,"jax":155},[154],[157,6092,6093,6107],{"style":606,"xmlns":160,"width":607,"height":608,"role":163,"focusable":164,"viewBox":609,"xmlnsXLink":166},[168,6094,6095,6098,6101,6104],{},[171,6096],{"id":6097,"d":178},"MJX-57-TEX-I-1D465",[171,6099],{"id":6100,"d":174},"MJX-57-TEX-N-28",[171,6102],{"id":6103,"d":190},"MJX-57-TEX-I-1D456",[171,6105],{"id":6106,"d":198},"MJX-57-TEX-N-29",[208,6108,6109],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,6110,6111],{"dataMmlNode":215},[208,6112,6113,6118],{"dataMmlNode":630},[208,6114,6115],{"dataMmlNode":226},[220,6116],{"dataC":230,"xLinkHref":6117},"#MJX-57-TEX-I-1D465",[208,6119,6120,6125,6130],{"dataMmlNode":269,"transform":638,"dataMjxTexclass":270},[208,6121,6122],{"dataMmlNode":218},[220,6123],{"dataC":222,"xLinkHref":6124},"#MJX-57-TEX-N-28",[208,6126,6127],{"dataMmlNode":226,"transform":227},[220,6128],{"dataC":251,"xLinkHref":6129},"#MJX-57-TEX-I-1D456",[208,6131,6132],{"dataMmlNode":218,"transform":651},[220,6133],{"dataC":265,"xLinkHref":6134},"#MJX-57-TEX-N-29"," and the specific point you are trying to predict ",[151,6137,6139],{"className":6138,"jax":155},[154],[157,6140,6141,6146],{"style":606,"xmlns":160,"width":5730,"height":825,"role":163,"focusable":164,"viewBox":5731,"xmlnsXLink":166},[168,6142,6143],{},[171,6144],{"id":6145,"d":178},"MJX-58-TEX-I-1D465",[208,6147,6148],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,6149,6150],{"dataMmlNode":215},[208,6151,6152],{"dataMmlNode":226},[220,6153],{"dataC":230,"xLinkHref":6154},"#MJX-58-TEX-I-1D465",[520,6156,6157,6162],{},[523,6158,6159,6160,1542],{},"If the training point is very close to your query point, this value is almost ",[20,6161,211],{},[523,6163,6164,6165,1542],{},"If the training point is very far away, this value is ",[20,6166,6167],{},"large",[523,6169,6170,1498,6173,6250,6252,6253,6274,6275],{},[20,6171,6172],{},"The Exponential & Negative Sign:",[151,6174,6176],{"className":6175,"jax":155},[154],[157,6177,6180,6206],{"style":159,"xmlns":160,"width":6178,"height":162,"role":163,"focusable":164,"viewBox":6179,"xmlnsXLink":166},"10.006ex","0 -750 4422.7 1000",[168,6181,6182,6185,6188,6191,6194,6197,6200,6203],{},[171,6183],{"id":6184,"d":5484},"MJX-59-TEX-N-65",[171,6186],{"id":6187,"d":5488},"MJX-59-TEX-N-78",[171,6189],{"id":6190,"d":5492},"MJX-59-TEX-N-70",[171,6192],{"id":6193,"d":5496},"MJX-59-TEX-N-2061",[171,6195],{"id":6196,"d":174},"MJX-59-TEX-N-28",[171,6198],{"id":6199,"d":182},"MJX-59-TEX-N-2212",[171,6201],{"id":6202,"d":2599},"MJX-59-TEX-N-2026",[171,6204],{"id":6205,"d":198},"MJX-59-TEX-N-29",[208,6207,6208],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,6209,6210,6221,6227,6232,6238,6244],{"dataMmlNode":215},[208,6211,6212,6215,6218],{"dataMmlNode":226},[220,6213],{"dataC":5557,"xLinkHref":6214},"#MJX-59-TEX-N-65",[220,6216],{"dataC":5561,"xLinkHref":6217,"transform":5563},"#MJX-59-TEX-N-78",[220,6219],{"dataC":5566,"xLinkHref":6220,"transform":5568},"#MJX-59-TEX-N-70",[208,6222,6224],{"dataMmlNode":218,"transform":6223},"translate(1528,0)",[220,6225],{"dataC":5574,"xLinkHref":6226},"#MJX-59-TEX-N-2061",[208,6228,6229],{"dataMmlNode":218,"transform":6223},[220,6230],{"dataC":222,"xLinkHref":6231},"#MJX-59-TEX-N-28",[208,6233,6235],{"dataMmlNode":218,"transform":6234},"translate(1917,0)",[220,6236],{"dataC":237,"xLinkHref":6237},"#MJX-59-TEX-N-2212",[208,6239,6241],{"dataMmlNode":218,"transform":6240},"translate(2861.7,0)",[220,6242],{"dataC":2811,"xLinkHref":6243},"#MJX-59-TEX-N-2026",[208,6245,6247],{"dataMmlNode":218,"transform":6246},"translate(4033.7,0)",[220,6248],{"dataC":265,"xLinkHref":6249},"#MJX-59-TEX-N-29",[6085,6251],{},"Because of the negative sign, we are doing ",[151,6254,6256],{"className":6255,"jax":155},[154],[157,6257,6260,6265],{"style":606,"xmlns":160,"width":6258,"height":825,"role":163,"focusable":164,"viewBox":6259,"xmlnsXLink":166},"1.054ex","0 -442 466 453",[168,6261,6262],{},[171,6263],{"id":6264,"d":354},"MJX-60-TEX-I-1D452",[208,6266,6267],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,6268,6269],{"dataMmlNode":215},[208,6270,6271],{"dataMmlNode":226},[220,6272],{"dataC":415,"xLinkHref":6273},"#MJX-60-TEX-I-1D452"," to the power of a negative number.",[520,6276,6277,6375],{},[523,6278,6279,6280,6299,6300,6351,6352,6371,6372,1542],{},"If the distance is roughly ",[151,6281,6283],{"className":6282,"jax":155},[154],[157,6284,6285,6290],{"style":3975,"xmlns":160,"width":3337,"height":3976,"role":163,"focusable":164,"viewBox":3977,"xmlnsXLink":166},[168,6286,6287],{},[171,6288],{"id":6289,"d":1047},"MJX-61-TEX-N-30",[208,6291,6292],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,6293,6294],{"dataMmlNode":215},[208,6295,6296],{"dataMmlNode":1103},[220,6297],{"dataC":1107,"xLinkHref":6298},"#MJX-61-TEX-N-30"," (the points are close), we get ",[151,6301,6303],{"className":6302,"jax":155},[154],[157,6304,6308,6322],{"style":1888,"xmlns":160,"width":6305,"height":6306,"role":163,"focusable":164,"viewBox":6307,"xmlnsXLink":166},"6.19ex","2.072ex","0 -833.9 2736.1 915.9",[168,6309,6310,6313,6316,6319],{},[171,6311],{"id":6312,"d":354},"MJX-62-TEX-I-1D452",[171,6314],{"id":6315,"d":1047},"MJX-62-TEX-N-30",[171,6317],{"id":6318,"d":1039},"MJX-62-TEX-N-3D",[171,6320],{"id":6321,"d":1055},"MJX-62-TEX-N-31",[208,6323,6324],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,6325,6326,6339,6345],{"dataMmlNode":215},[208,6327,6328,6333],{"dataMmlNode":630},[208,6329,6330],{"dataMmlNode":226},[220,6331],{"dataC":415,"xLinkHref":6332},"#MJX-62-TEX-I-1D452",[208,6334,6336],{"dataMmlNode":1103,"transform":6335},"translate(499,363) scale(0.707)",[220,6337],{"dataC":1107,"xLinkHref":6338},"#MJX-62-TEX-N-30",[208,6340,6342],{"dataMmlNode":218,"transform":6341},"translate(1180.3,0)",[220,6343],{"dataC":1089,"xLinkHref":6344},"#MJX-62-TEX-N-3D",[208,6346,6348],{"dataMmlNode":1103,"transform":6347},"translate(2236.1,0)",[220,6349],{"dataC":1127,"xLinkHref":6350},"#MJX-62-TEX-N-31",". The weight is ",[151,6353,6355],{"className":6354,"jax":155},[154],[157,6356,6357,6362],{"style":3336,"xmlns":160,"width":3337,"height":3338,"role":163,"focusable":164,"viewBox":3339,"xmlnsXLink":166},[168,6358,6359],{},[171,6360],{"id":6361,"d":1055},"MJX-63-TEX-N-31",[208,6363,6364],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,6365,6366],{"dataMmlNode":215},[208,6367,6368],{"dataMmlNode":1103},[220,6369],{"dataC":1127,"xLinkHref":6370},"#MJX-63-TEX-N-31",". The model gives this point ",[20,6373,6374],{},"maximum importance",[523,6376,6377,6378,6456,6457,6459,6460,6479,6480,6483],{},"If the distance is large (the points are far), we get ",[151,6379,6381],{"className":6380,"jax":155},[154],[157,6382,6386,6413],{"style":606,"xmlns":160,"width":6383,"height":6384,"role":163,"focusable":164,"viewBox":6385,"xmlnsXLink":166},"5.869ex","1.956ex","0 -853.7 2594 864.7",[168,6387,6388,6391,6394,6398,6402,6406,6410],{},[171,6389],{"id":6390,"d":354},"MJX-64-TEX-I-1D452",[171,6392],{"id":6393,"d":182},"MJX-64-TEX-N-2212",[171,6395],{"id":6396,"d":6397},"MJX-64-TEX-N-6C","M42 46H56Q95 46 103 60V68Q103 77 103 91T103 124T104 167T104 217T104 272T104 329Q104 366 104 407T104 482T104 542T103 586T103 603Q100 622 89 628T44 637H26V660Q26 683 28 683L38 684Q48 685 67 686T104 688Q121 689 141 690T171 693T182 694H185V379Q185 62 186 60Q190 52 198 49Q219 46 247 46H263V0H255L232 1Q209 2 183 2T145 3T107 3T57 1L34 0H26V46H42Z",[171,6399],{"id":6400,"d":6401},"MJX-64-TEX-N-61","M137 305T115 305T78 320T63 359Q63 394 97 421T218 448Q291 448 336 416T396 340Q401 326 401 309T402 194V124Q402 76 407 58T428 40Q443 40 448 56T453 109V145H493V106Q492 66 490 59Q481 29 455 12T400 -6T353 12T329 54V58L327 55Q325 52 322 49T314 40T302 29T287 17T269 6T247 -2T221 -8T190 -11Q130 -11 82 20T34 107Q34 128 41 147T68 188T116 225T194 253T304 268H318V290Q318 324 312 340Q290 411 215 411Q197 411 181 410T156 406T148 403Q170 388 170 359Q170 334 154 320ZM126 106Q126 75 150 51T209 26Q247 26 276 49T315 109Q317 116 318 175Q318 233 317 233Q309 233 296 232T251 223T193 203T147 166T126 106Z",[171,6403],{"id":6404,"d":6405},"MJX-64-TEX-N-72","M36 46H50Q89 46 97 60V68Q97 77 97 91T98 122T98 161T98 203Q98 234 98 269T98 328L97 351Q94 370 83 376T38 385H20V408Q20 431 22 431L32 432Q42 433 60 434T96 436Q112 437 131 438T160 441T171 442H174V373Q213 441 271 441H277Q322 441 343 419T364 373Q364 352 351 337T313 322Q288 322 276 338T263 372Q263 381 265 388T270 400T273 405Q271 407 250 401Q234 393 226 386Q179 341 179 207V154Q179 141 179 127T179 101T180 81T180 66V61Q181 59 183 57T188 54T193 51T200 49T207 48T216 47T225 47T235 46T245 46H276V0H267Q249 3 140 3Q37 3 28 0H20V46H36Z",[171,6407],{"id":6408,"d":6409},"MJX-64-TEX-N-67","M329 409Q373 453 429 453Q459 453 472 434T485 396Q485 382 476 371T449 360Q416 360 412 390Q410 404 415 411Q415 412 416 414V415Q388 412 363 393Q355 388 355 386Q355 385 359 381T368 369T379 351T388 325T392 292Q392 230 343 187T222 143Q172 143 123 171Q112 153 112 133Q112 98 138 81Q147 75 155 75T227 73Q311 72 335 67Q396 58 431 26Q470 -13 470 -72Q470 -139 392 -175Q332 -206 250 -206Q167 -206 107 -175Q29 -140 29 -75Q29 -39 50 -15T92 18L103 24Q67 55 67 108Q67 155 96 193Q52 237 52 292Q52 355 102 398T223 442Q274 442 318 416L329 409ZM299 343Q294 371 273 387T221 404Q192 404 171 388T145 343Q142 326 142 292Q142 248 149 227T179 192Q196 182 222 182Q244 182 260 189T283 207T294 227T299 242Q302 258 302 292T299 343ZM403 -75Q403 -50 389 -34T348 -11T299 -2T245 0H218Q151 0 138 -6Q118 -15 107 -34T95 -74Q95 -84 101 -97T122 -127T170 -155T250 -167Q319 -167 361 -139T403 -75Z",[171,6411],{"id":6412,"d":5484},"MJX-64-TEX-N-65",[208,6414,6415],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,6416,6417],{"dataMmlNode":215},[208,6418,6419,6424],{"dataMmlNode":630},[208,6420,6421],{"dataMmlNode":226},[220,6422],{"dataC":415,"xLinkHref":6423},"#MJX-64-TEX-I-1D452",[208,6425,6426,6431],{"dataMmlNode":269,"transform":6335,"dataMjxTexclass":270},[208,6427,6428],{"dataMmlNode":218},[220,6429],{"dataC":237,"xLinkHref":6430},"#MJX-64-TEX-N-2212",[208,6432,6435,6439,6443,6447,6452],{"dataMmlNode":6433,"transform":6434},"mtext","translate(778,0)",[220,6436],{"dataC":6437,"xLinkHref":6438},"6C","#MJX-64-TEX-N-6C",[220,6440],{"dataC":6441,"xLinkHref":6442,"transform":1788},"61","#MJX-64-TEX-N-61",[220,6444],{"dataC":6445,"xLinkHref":6446,"transform":6434},"72","#MJX-64-TEX-N-72",[220,6448],{"dataC":6449,"xLinkHref":6450,"transform":6451},"67","#MJX-64-TEX-N-67","translate(1170,0)",[220,6453],{"dataC":5557,"xLinkHref":6454,"transform":6455},"#MJX-64-TEX-N-65","translate(1670,0)",", which approaches ",[20,6458,211],{},". The weight becomes ",[151,6461,6463],{"className":6462,"jax":155},[154],[157,6464,6465,6470],{"style":3975,"xmlns":160,"width":3337,"height":3976,"role":163,"focusable":164,"viewBox":3977,"xmlnsXLink":166},[168,6466,6467],{},[171,6468],{"id":6469,"d":1047},"MJX-65-TEX-N-30",[208,6471,6472],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,6473,6474],{"dataMmlNode":215},[208,6475,6476],{"dataMmlNode":1103},[220,6477],{"dataC":1107,"xLinkHref":6478},"#MJX-65-TEX-N-30",". The model completely ",[20,6481,6482],{},"ignores"," this point.",[523,6485,6486,1498,6489,6508,6509,6511,6530,6531,1542,6550],{},[20,6487,6488],{},"The Bandwidth Parameter:",[151,6490,6492],{"className":6491,"jax":155},[154],[157,6493,6494,6499],{"style":5679,"xmlns":160,"width":5680,"height":5681,"role":163,"focusable":164,"viewBox":5682,"xmlnsXLink":166},[168,6495,6496],{},[171,6497],{"id":6498,"d":5512},"MJX-66-TEX-I-1D70F",[208,6500,6501],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,6502,6503],{"dataMmlNode":215},[208,6504,6505],{"dataMmlNode":226},[220,6506],{"dataC":5655,"xLinkHref":6507},"#MJX-66-TEX-I-1D70F"," (Tau)",[6085,6510],{},[151,6512,6514],{"className":6513,"jax":155},[154],[157,6515,6516,6521],{"style":5679,"xmlns":160,"width":5680,"height":5681,"role":163,"focusable":164,"viewBox":5682,"xmlnsXLink":166},[168,6517,6518],{},[171,6519],{"id":6520,"d":5512},"MJX-67-TEX-I-1D70F",[208,6522,6523],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,6524,6525],{"dataMmlNode":215},[208,6526,6527],{"dataMmlNode":226},[220,6528],{"dataC":5655,"xLinkHref":6529},"#MJX-67-TEX-I-1D70F"," dictates how \"fat\" or \"thin\" our bell curve is. It controls how quickly the weights fall off to zero as you move away from the query point ",[151,6532,6534],{"className":6533,"jax":155},[154],[157,6535,6536,6541],{"style":606,"xmlns":160,"width":5730,"height":825,"role":163,"focusable":164,"viewBox":5731,"xmlnsXLink":166},[168,6537,6538],{},[171,6539],{"id":6540,"d":178},"MJX-68-TEX-I-1D465",[208,6542,6543],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,6544,6545],{"dataMmlNode":215},[208,6546,6547],{"dataMmlNode":226},[220,6548],{"dataC":230,"xLinkHref":6549},"#MJX-68-TEX-I-1D465",[520,6551,6552,6599],{},[523,6553,6554,1498,6557,6578,6579,6598],{},[20,6555,6556],{},"Large",[20,6558,6559,4049],{},[151,6560,6562],{"className":6561,"jax":155},[154],[157,6563,6564,6569],{"style":5679,"xmlns":160,"width":5680,"height":5681,"role":163,"focusable":164,"viewBox":5682,"xmlnsXLink":166},[168,6565,6566],{},[171,6567],{"id":6568,"d":5512},"MJX-69-TEX-I-1D70F",[208,6570,6571],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,6572,6573],{"dataMmlNode":215},[208,6574,6575],{"dataMmlNode":226},[220,6576],{"dataC":5655,"xLinkHref":6577},"#MJX-69-TEX-I-1D70F"," The weights fall off very slowly. The model looks at a very wide \"neighborhood\" of points. If ",[151,6580,6582],{"className":6581,"jax":155},[154],[157,6583,6584,6589],{"style":5679,"xmlns":160,"width":5680,"height":5681,"role":163,"focusable":164,"viewBox":5682,"xmlnsXLink":166},[168,6585,6586],{},[171,6587],{"id":6588,"d":5512},"MJX-70-TEX-I-1D70F",[208,6590,6591],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,6592,6593],{"dataMmlNode":215},[208,6594,6595],{"dataMmlNode":226},[220,6596],{"dataC":5655,"xLinkHref":6597},"#MJX-70-TEX-I-1D70F"," is infinitely large, LWLR just becomes standard linear regression.",[523,6600,6601,1498,6604,6625,6626,1542],{},[20,6602,6603],{},"Small",[20,6605,6606,4049],{},[151,6607,6609],{"className":6608,"jax":155},[154],[157,6610,6611,6616],{"style":5679,"xmlns":160,"width":5680,"height":5681,"role":163,"focusable":164,"viewBox":5682,"xmlnsXLink":166},[168,6612,6613],{},[171,6614],{"id":6615,"d":5512},"MJX-71-TEX-I-1D70F",[208,6617,6618],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,6619,6620],{"dataMmlNode":215},[208,6621,6622],{"dataMmlNode":226},[220,6623],{"dataC":5655,"xLinkHref":6624},"#MJX-71-TEX-I-1D70F"," The weights fall off rapidly. The model only looks at a very tiny, strict neighborhood of points immediately next to ",[151,6627,6629],{"className":6628,"jax":155},[154],[157,6630,6631,6636],{"style":606,"xmlns":160,"width":5730,"height":825,"role":163,"focusable":164,"viewBox":5731,"xmlnsXLink":166},[168,6632,6633],{},[171,6634],{"id":6635,"d":178},"MJX-72-TEX-I-1D465",[208,6637,6638],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,6639,6640],{"dataMmlNode":215},[208,6641,6642],{"dataMmlNode":226},[220,6643],{"dataC":230,"xLinkHref":6644},"#MJX-72-TEX-I-1D465",[28,6646,6648],{"id":6647},"why-is-it-called-non-parametric","Why is it called \"Non-parametric\"?",[16,6650,6651,6652,6655,6656,6675,6676,6773],{},"Standard linear regression is ",[20,6653,6654],{},"parametric",". You look at the data once, calculate your ",[151,6657,6659],{"className":6658,"jax":155},[154],[157,6660,6661,6666],{"style":1178,"xmlns":160,"width":1179,"height":1180,"role":163,"focusable":164,"viewBox":1181,"xmlnsXLink":166},[168,6662,6663],{},[171,6664],{"id":6665,"d":1043},"MJX-73-TEX-I-1D703",[208,6667,6668],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,6669,6670],{"dataMmlNode":215},[208,6671,6672],{"dataMmlNode":226},[220,6673],{"dataC":1099,"xLinkHref":6674},"#MJX-73-TEX-I-1D703"," parameters, and then you can throw the training data away forever. Your formula ",[151,6677,6679],{"className":6678,"jax":155},[154],[157,6680,6683,6712],{"style":159,"xmlns":160,"width":6681,"height":162,"role":163,"focusable":164,"viewBox":6682,"xmlnsXLink":166},"15.532ex","0 -750 6865.1 1000",[168,6684,6685,6688,6691,6694,6697,6700,6703,6706,6709],{},[171,6686],{"id":6687,"d":858},"MJX-74-TEX-I-210E",[171,6689],{"id":6690,"d":174},"MJX-74-TEX-N-28",[171,6692],{"id":6693,"d":178},"MJX-74-TEX-I-1D465",[171,6695],{"id":6696,"d":198},"MJX-74-TEX-N-29",[171,6698],{"id":6699,"d":1039},"MJX-74-TEX-N-3D",[171,6701],{"id":6702,"d":1043},"MJX-74-TEX-I-1D703",[171,6704],{"id":6705,"d":1047},"MJX-74-TEX-N-30",[171,6707],{"id":6708,"d":1051},"MJX-74-TEX-N-2B",[171,6710],{"id":6711,"d":1055},"MJX-74-TEX-N-31",[208,6713,6714],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,6715,6716,6721,6726,6731,6736,6741,6753,6758,6769],{"dataMmlNode":215},[208,6717,6718],{"dataMmlNode":226},[220,6719],{"dataC":876,"xLinkHref":6720},"#MJX-74-TEX-I-210E",[208,6722,6723],{"dataMmlNode":218,"transform":880},[220,6724],{"dataC":222,"xLinkHref":6725},"#MJX-74-TEX-N-28",[208,6727,6728],{"dataMmlNode":226,"transform":886},[220,6729],{"dataC":230,"xLinkHref":6730},"#MJX-74-TEX-I-1D465",[208,6732,6733],{"dataMmlNode":218,"transform":892},[220,6734],{"dataC":265,"xLinkHref":6735},"#MJX-74-TEX-N-29",[208,6737,6738],{"dataMmlNode":218,"transform":1086},[220,6739],{"dataC":1089,"xLinkHref":6740},"#MJX-74-TEX-N-3D",[208,6742,6743,6748],{"dataMmlNode":1093,"transform":1094},[208,6744,6745],{"dataMmlNode":226},[220,6746],{"dataC":1099,"xLinkHref":6747},"#MJX-74-TEX-I-1D703",[208,6749,6750],{"dataMmlNode":1103,"transform":1104},[220,6751],{"dataC":1107,"xLinkHref":6752},"#MJX-74-TEX-N-30",[208,6754,6755],{"dataMmlNode":218,"transform":1111},[220,6756],{"dataC":1114,"xLinkHref":6757},"#MJX-74-TEX-N-2B",[208,6759,6760,6764],{"dataMmlNode":1093,"transform":1118},[208,6761,6762],{"dataMmlNode":226},[220,6763],{"dataC":1099,"xLinkHref":6747},[208,6765,6766],{"dataMmlNode":1103,"transform":1104},[220,6767],{"dataC":1127,"xLinkHref":6768},"#MJX-74-TEX-N-31",[208,6770,6771],{"dataMmlNode":226,"transform":1131},[220,6772],{"dataC":230,"xLinkHref":6730}," does all the work.",[16,6775,6776,6777,6779,6780,6827,6828,6847,6848,1498,6851,1498,6872,6875,6876,6895,6896,6915,6916,1542],{},"LWLR is ",[20,6778,5700],{},". Because the weights ",[151,6781,6783],{"className":6782,"jax":155},[154],[157,6784,6785,6799],{"style":606,"xmlns":160,"width":5758,"height":608,"role":163,"focusable":164,"viewBox":5759,"xmlnsXLink":166},[168,6786,6787,6790,6793,6796],{},[171,6788],{"id":6789,"d":5468},"MJX-75-TEX-I-1D464",[171,6791],{"id":6792,"d":174},"MJX-75-TEX-N-28",[171,6794],{"id":6795,"d":190},"MJX-75-TEX-I-1D456",[171,6797],{"id":6798,"d":198},"MJX-75-TEX-N-29",[208,6800,6801],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,6802,6803],{"dataMmlNode":215},[208,6804,6805,6810],{"dataMmlNode":630},[208,6806,6807],{"dataMmlNode":226},[220,6808],{"dataC":5526,"xLinkHref":6809},"#MJX-75-TEX-I-1D464",[208,6811,6812,6817,6822],{"dataMmlNode":269,"transform":5787,"dataMjxTexclass":270},[208,6813,6814],{"dataMmlNode":218},[220,6815],{"dataC":222,"xLinkHref":6816},"#MJX-75-TEX-N-28",[208,6818,6819],{"dataMmlNode":226,"transform":227},[220,6820],{"dataC":251,"xLinkHref":6821},"#MJX-75-TEX-I-1D456",[208,6823,6824],{"dataMmlNode":218,"transform":651},[220,6825],{"dataC":265,"xLinkHref":6826},"#MJX-75-TEX-N-29"," depend on ",[151,6829,6831],{"className":6830,"jax":155},[154],[157,6832,6833,6838],{"style":606,"xmlns":160,"width":5730,"height":825,"role":163,"focusable":164,"viewBox":5731,"xmlnsXLink":166},[168,6834,6835],{},[171,6836],{"id":6837,"d":178},"MJX-76-TEX-I-1D465",[208,6839,6840],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,6841,6842],{"dataMmlNode":215},[208,6843,6844],{"dataMmlNode":226},[220,6845],{"dataC":230,"xLinkHref":6846},"#MJX-76-TEX-I-1D465"," (the specific thing you are trying to predict right now, ",[20,6849,6850],{},"you must recalculate",[20,6852,6853],{},[151,6854,6856],{"className":6855,"jax":155},[154],[157,6857,6858,6863],{"style":1178,"xmlns":160,"width":1179,"height":1180,"role":163,"focusable":164,"viewBox":1181,"xmlnsXLink":166},[168,6859,6860],{},[171,6861],{"id":6862,"d":1043},"MJX-77-TEX-I-1D703",[208,6864,6865],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,6866,6867],{"dataMmlNode":215},[208,6868,6869],{"dataMmlNode":226},[220,6870],{"dataC":1099,"xLinkHref":6871},"#MJX-77-TEX-I-1D703",[20,6873,6874],{},"from scratch every single time you want to make a new prediction.","), you cannot pre-calculate your ",[151,6877,6879],{"className":6878,"jax":155},[154],[157,6880,6881,6886],{"style":1178,"xmlns":160,"width":1179,"height":1180,"role":163,"focusable":164,"viewBox":1181,"xmlnsXLink":166},[168,6882,6883],{},[171,6884],{"id":6885,"d":1043},"MJX-78-TEX-I-1D703",[208,6887,6888],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,6889,6890],{"dataMmlNode":215},[208,6891,6892],{"dataMmlNode":226},[220,6893],{"dataC":1099,"xLinkHref":6894},"#MJX-78-TEX-I-1D703"," values. You have to keep the entire training dataset in memory and calculate a brand new set of ",[151,6897,6899],{"className":6898,"jax":155},[154],[157,6900,6901,6906],{"style":1178,"xmlns":160,"width":1179,"height":1180,"role":163,"focusable":164,"viewBox":1181,"xmlnsXLink":166},[168,6902,6903],{},[171,6904],{"id":6905,"d":1043},"MJX-79-TEX-I-1D703",[208,6907,6908],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,6909,6910],{"dataMmlNode":215},[208,6911,6912],{"dataMmlNode":226},[220,6913],{"dataC":1099,"xLinkHref":6914},"#MJX-79-TEX-I-1D703"," values for ",[20,6917,6918],{},"every single prediction you make",[11,6920,6922],{"id":6921},"logistic-regression","Logistic Regression",[6085,6924],{},[16,6926,6927,6928,6931],{},"Everything above assumes y is continuous. But what if y can only be ",[20,6929,6930],{},"0 or 1"," (e.g., spam vs. not spam, tumor is malignant vs. benign)?",[16,6933,6934,6935,6939,6940,4049],{},"We can't use a plain linear function because it can output values outside ",[6936,6937,6938],"span",{},"0, 1",". Instead, we wrap it in the ",[20,6941,6942],{},"sigmoid (logistic) function",[151,6944,6946],{"className":6945,"jax":155,"display":1017},[154],[157,6947,6952,6988],{"style":6948,"xmlns":160,"width":6949,"height":6950,"role":163,"focusable":164,"viewBox":6951,"xmlnsXLink":166},"vertical-align: -1.977ex;","18.286ex","5.013ex","0 -1342 8082.6 2215.9",[168,6953,6954,6957,6960,6963,6966,6969,6972,6975,6978,6981,6984],{},[171,6955],{"id":6956,"d":858},"MJX-80-TEX-I-210E",[171,6958],{"id":6959,"d":1043},"MJX-80-TEX-I-1D703",[171,6961],{"id":6962,"d":174},"MJX-80-TEX-N-28",[171,6964],{"id":6965,"d":178},"MJX-80-TEX-I-1D465",[171,6967],{"id":6968,"d":198},"MJX-80-TEX-N-29",[171,6970],{"id":6971,"d":1039},"MJX-80-TEX-N-3D",[171,6973],{"id":6974,"d":1055},"MJX-80-TEX-N-31",[171,6976],{"id":6977,"d":1051},"MJX-80-TEX-N-2B",[171,6979],{"id":6980,"d":354},"MJX-80-TEX-I-1D452",[171,6982],{"id":6983,"d":182},"MJX-80-TEX-N-2212",[171,6985],{"id":6986,"d":6987},"MJX-80-TEX-I-1D447","M40 437Q21 437 21 445Q21 450 37 501T71 602L88 651Q93 669 101 677H569H659Q691 677 697 676T704 667Q704 661 687 553T668 444Q668 437 649 437Q640 437 637 437T631 442L629 445Q629 451 635 490T641 551Q641 586 628 604T573 629Q568 630 515 631Q469 631 457 630T439 622Q438 621 368 343T298 60Q298 48 386 46Q418 46 427 45T436 36Q436 31 433 22Q429 4 424 1L422 0Q419 0 415 0Q410 0 363 1T228 2Q99 2 64 0H49Q43 6 43 9T45 27Q49 40 55 46H83H94Q174 46 189 55Q190 56 191 56Q196 59 201 76T241 233Q258 301 269 344Q339 619 339 625Q339 630 310 630H279Q212 630 191 624Q146 614 121 583T67 467Q60 445 57 441T43 437H40Z",[208,6989,6990],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,6991,6992,7005,7011,7017,7023,7029],{"dataMmlNode":215},[208,6993,6994,6999],{"dataMmlNode":1093},[208,6995,6996],{"dataMmlNode":226},[220,6997],{"dataC":876,"xLinkHref":6998},"#MJX-80-TEX-I-210E",[208,7000,7002],{"dataMmlNode":226,"transform":7001},"translate(609,-150) scale(0.707)",[220,7003],{"dataC":1099,"xLinkHref":7004},"#MJX-80-TEX-I-1D703",[208,7006,7008],{"dataMmlNode":218,"transform":7007},"translate(990.6,0)",[220,7009],{"dataC":222,"xLinkHref":7010},"#MJX-80-TEX-N-28",[208,7012,7014],{"dataMmlNode":226,"transform":7013},"translate(1379.6,0)",[220,7015],{"dataC":230,"xLinkHref":7016},"#MJX-80-TEX-I-1D465",[208,7018,7020],{"dataMmlNode":218,"transform":7019},"translate(1951.6,0)",[220,7021],{"dataC":265,"xLinkHref":7022},"#MJX-80-TEX-N-29",[208,7024,7026],{"dataMmlNode":218,"transform":7025},"translate(2618.4,0)",[220,7027],{"dataC":1089,"xLinkHref":7028},"#MJX-80-TEX-N-3D",[208,7030,7032,7038,7085],{"dataMmlNode":1324,"transform":7031},"translate(3674.2,0)",[208,7033,7035],{"dataMmlNode":1103,"transform":7034},"translate(1954.2,676)",[220,7036],{"dataC":1127,"xLinkHref":7037},"#MJX-80-TEX-N-31",[208,7039,7041,7045,7051],{"dataMmlNode":1821,"transform":7040},"translate(220,-791.9)",[208,7042,7043],{"dataMmlNode":1103},[220,7044],{"dataC":1127,"xLinkHref":7037},[208,7046,7048],{"dataMmlNode":218,"transform":7047},"translate(722.2,0)",[220,7049],{"dataC":1114,"xLinkHref":7050},"#MJX-80-TEX-N-2B",[208,7052,7054,7059],{"dataMmlNode":630,"transform":7053},"translate(1722.4,0)",[208,7055,7056],{"dataMmlNode":226},[220,7057],{"dataC":415,"xLinkHref":7058},"#MJX-80-TEX-I-1D452",[208,7060,7062,7067,7080],{"dataMmlNode":269,"transform":7061,"dataMjxTexclass":270},"translate(499,289) scale(0.707)",[208,7063,7064],{"dataMmlNode":218},[220,7065],{"dataC":237,"xLinkHref":7066},"#MJX-80-TEX-N-2212",[208,7068,7069,7073],{"dataMmlNode":630,"transform":6434},[208,7070,7071],{"dataMmlNode":226},[220,7072],{"dataC":1099,"xLinkHref":7004},[208,7074,7076],{"dataMmlNode":226,"transform":7075},"translate(502,289) scale(0.707)",[220,7077],{"dataC":7078,"xLinkHref":7079},"1D447","#MJX-80-TEX-I-1D447",[208,7081,7083],{"dataMmlNode":226,"transform":7082},"translate(1827.8,0)",[220,7084],{"dataC":230,"xLinkHref":7016},[1339,7086],{"width":7087,"height":1342,"x":1343,"y":1344},4168.4,[16,7089,7090,7091,7170],{},"The sigmoid squashes any real number into the range (0, 1), which we interpret as a probability. If ",[151,7092,7094],{"className":7093,"jax":155},[154],[157,7095,7098,7127],{"style":159,"xmlns":160,"width":7096,"height":162,"role":163,"focusable":164,"viewBox":7097,"xmlnsXLink":166},"10.266ex","0 -750 4537.6 1000",[168,7099,7100,7103,7106,7109,7112,7116,7119,7123],{},[171,7101],{"id":7102,"d":858},"MJX-81-TEX-I-210E",[171,7104],{"id":7105,"d":174},"MJX-81-TEX-N-28",[171,7107],{"id":7108,"d":178},"MJX-81-TEX-I-1D465",[171,7110],{"id":7111,"d":198},"MJX-81-TEX-N-29",[171,7113],{"id":7114,"d":7115},"MJX-81-TEX-N-2265","M83 616Q83 624 89 630T99 636Q107 636 253 568T543 431T687 361Q694 356 694 346T687 331Q685 329 395 192L107 56H101Q83 58 83 76Q83 77 83 79Q82 86 98 95Q117 105 248 167Q326 204 378 228L626 346L360 472Q291 505 200 548Q112 589 98 597T83 616ZM84 -118Q84 -108 99 -98H678Q694 -104 694 -118Q694 -130 679 -138H98Q84 -131 84 -118Z",[171,7117],{"id":7118,"d":1047},"MJX-81-TEX-N-30",[171,7120],{"id":7121,"d":7122},"MJX-81-TEX-N-2E","M78 60Q78 84 95 102T138 120Q162 120 180 104T199 61Q199 36 182 18T139 0T96 17T78 60Z",[171,7124],{"id":7125,"d":7126},"MJX-81-TEX-N-35","M164 157Q164 133 148 117T109 101H102Q148 22 224 22Q294 22 326 82Q345 115 345 210Q345 313 318 349Q292 382 260 382H254Q176 382 136 314Q132 307 129 306T114 304Q97 304 95 310Q93 314 93 485V614Q93 664 98 664Q100 666 102 666Q103 666 123 658T178 642T253 634Q324 634 389 662Q397 666 402 666Q410 666 410 648V635Q328 538 205 538Q174 538 149 544L139 546V374Q158 388 169 396T205 412T256 420Q337 420 393 355T449 201Q449 109 385 44T229 -22Q148 -22 99 32T50 154Q50 178 61 192T84 210T107 214Q132 214 148 197T164 157Z",[208,7128,7129],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,7130,7131,7136,7141,7146,7151,7157],{"dataMmlNode":215},[208,7132,7133],{"dataMmlNode":226},[220,7134],{"dataC":876,"xLinkHref":7135},"#MJX-81-TEX-I-210E",[208,7137,7138],{"dataMmlNode":218,"transform":880},[220,7139],{"dataC":222,"xLinkHref":7140},"#MJX-81-TEX-N-28",[208,7142,7143],{"dataMmlNode":226,"transform":886},[220,7144],{"dataC":230,"xLinkHref":7145},"#MJX-81-TEX-I-1D465",[208,7147,7148],{"dataMmlNode":218,"transform":892},[220,7149],{"dataC":265,"xLinkHref":7150},"#MJX-81-TEX-N-29",[208,7152,7153],{"dataMmlNode":218,"transform":1086},[220,7154],{"dataC":7155,"xLinkHref":7156},"2265","#MJX-81-TEX-N-2265",[208,7158,7159,7162,7166],{"dataMmlNode":1103,"transform":1094},[220,7160],{"dataC":1107,"xLinkHref":7161},"#MJX-81-TEX-N-30",[220,7163],{"dataC":7164,"xLinkHref":7165,"transform":3275},"2E","#MJX-81-TEX-N-2E",[220,7167],{"dataC":7168,"xLinkHref":7169,"transform":6434},"35","#MJX-81-TEX-N-35",", we predict y = 1; otherwise, y = 0.",[69,7172,7174,7177],{"className":7173},[72,73],[75,7175,7176],{},"The Problem with Linear Regression for Classification",[79,7178,7180,7280,7289,7334,7448,7452,7495,7506,7643,7751,7965],{"className":7179},[82],[16,7181,7182,7183,7223,7224,7259,7260,7279],{},"Imagine you are a doctor trying to predict if a tumor is Malignant (",[151,7184,7186],{"className":7185,"jax":155},[154],[157,7187,7191,7202],{"style":665,"xmlns":160,"width":7188,"height":7189,"role":163,"focusable":164,"viewBox":7190,"xmlnsXLink":166},"5.257ex","1.971ex","0 -666 2323.6 871",[168,7192,7193,7196,7199],{},[171,7194],{"id":7195,"d":674},"MJX-82-TEX-I-1D466",[171,7197],{"id":7198,"d":1039},"MJX-82-TEX-N-3D",[171,7200],{"id":7201,"d":1055},"MJX-82-TEX-N-31",[208,7203,7204],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,7205,7206,7211,7217],{"dataMmlNode":215},[208,7207,7208],{"dataMmlNode":226},[220,7209],{"dataC":694,"xLinkHref":7210},"#MJX-82-TEX-I-1D466",[208,7212,7214],{"dataMmlNode":218,"transform":7213},"translate(767.8,0)",[220,7215],{"dataC":1089,"xLinkHref":7216},"#MJX-82-TEX-N-3D",[208,7218,7220],{"dataMmlNode":1103,"transform":7219},"translate(1823.6,0)",[220,7221],{"dataC":1127,"xLinkHref":7222},"#MJX-82-TEX-N-31",") or Benign (",[151,7225,7227],{"className":7226,"jax":155},[154],[157,7228,7229,7240],{"style":665,"xmlns":160,"width":7188,"height":7189,"role":163,"focusable":164,"viewBox":7190,"xmlnsXLink":166},[168,7230,7231,7234,7237],{},[171,7232],{"id":7233,"d":674},"MJX-83-TEX-I-1D466",[171,7235],{"id":7236,"d":1039},"MJX-83-TEX-N-3D",[171,7238],{"id":7239,"d":1047},"MJX-83-TEX-N-30",[208,7241,7242],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,7243,7244,7249,7254],{"dataMmlNode":215},[208,7245,7246],{"dataMmlNode":226},[220,7247],{"dataC":694,"xLinkHref":7248},"#MJX-83-TEX-I-1D466",[208,7250,7251],{"dataMmlNode":218,"transform":7213},[220,7252],{"dataC":1089,"xLinkHref":7253},"#MJX-83-TEX-N-3D",[208,7255,7256],{"dataMmlNode":1103,"transform":7219},[220,7257],{"dataC":1107,"xLinkHref":7258},"#MJX-83-TEX-N-30",") based on its size (",[151,7261,7263],{"className":7262,"jax":155},[154],[157,7264,7265,7270],{"style":606,"xmlns":160,"width":5730,"height":825,"role":163,"focusable":164,"viewBox":5731,"xmlnsXLink":166},[168,7266,7267],{},[171,7268],{"id":7269,"d":178},"MJX-84-TEX-I-1D465",[208,7271,7272],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,7273,7274],{"dataMmlNode":215},[208,7275,7276],{"dataMmlNode":226},[220,7277],{"dataC":230,"xLinkHref":7278},"#MJX-84-TEX-I-1D465",").",[16,7281,7282,7283,7285,7286],{},"If you use standard ",[20,7284,947],{},", you fit a straight line through the data. You might say: ",[87,7287,7288],{},"\"If the line outputs a value greater than 0.5, I will predict 1 (Malignant). If it's less than 0.5, I predict 0 (Benign).\"",[16,7290,7291,7292,7295,7296,7329,7330,7333],{},"This works okay until a patient comes in with a ",[20,7293,7294],{},"massive"," tumor.\nBecause linear regression is a straight line, it will try to accommodate this massive outlier. The line will tilt upwards heavily. Suddenly, your ",[151,7297,7299],{"className":7298,"jax":155},[154],[157,7300,7303,7314],{"style":3975,"xmlns":160,"width":7301,"height":3976,"role":163,"focusable":164,"viewBox":7302,"xmlnsXLink":166},"2.891ex","0 -666 1278 688",[168,7304,7305,7308,7311],{},[171,7306],{"id":7307,"d":1047},"MJX-85-TEX-N-30",[171,7309],{"id":7310,"d":7122},"MJX-85-TEX-N-2E",[171,7312],{"id":7313,"d":7126},"MJX-85-TEX-N-35",[208,7315,7316],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,7317,7318],{"dataMmlNode":215},[208,7319,7320,7323,7326],{"dataMmlNode":1103},[220,7321],{"dataC":1107,"xLinkHref":7322},"#MJX-85-TEX-N-30",[220,7324],{"dataC":7164,"xLinkHref":7325,"transform":3275},"#MJX-85-TEX-N-2E",[220,7327],{"dataC":7168,"xLinkHref":7328,"transform":6434},"#MJX-85-TEX-N-35"," threshold shifts, and tumors that you ",[87,7331,7332],{},"used"," to correctly predict as Malignant are now being predicted as Benign.",[16,7335,7336,7337,7388,7389,7447],{},"Even worse, the straight line might output a value like ",[151,7338,7340],{"className":7339,"jax":155},[154],[157,7341,7344,7362],{"style":665,"xmlns":160,"width":7342,"height":7189,"role":163,"focusable":164,"viewBox":7343,"xmlnsXLink":166},"7.017ex","0 -666 3101.6 871",[168,7345,7346,7349,7352,7356,7359],{},[171,7347],{"id":7348,"d":674},"MJX-86-TEX-I-1D466",[171,7350],{"id":7351,"d":1039},"MJX-86-TEX-N-3D",[171,7353],{"id":7354,"d":7355},"MJX-86-TEX-N-33","M127 463Q100 463 85 480T69 524Q69 579 117 622T233 665Q268 665 277 664Q351 652 390 611T430 522Q430 470 396 421T302 350L299 348Q299 347 308 345T337 336T375 315Q457 262 457 175Q457 96 395 37T238 -22Q158 -22 100 21T42 130Q42 158 60 175T105 193Q133 193 151 175T169 130Q169 119 166 110T159 94T148 82T136 74T126 70T118 67L114 66Q165 21 238 21Q293 21 321 74Q338 107 338 175V195Q338 290 274 322Q259 328 213 329L171 330L168 332Q166 335 166 348Q166 366 174 366Q202 366 232 371Q266 376 294 413T322 525V533Q322 590 287 612Q265 626 240 626Q208 626 181 615T143 592T132 580H135Q138 579 143 578T153 573T165 566T175 555T183 540T186 520Q186 498 172 481T127 463Z",[171,7357],{"id":7358,"d":7122},"MJX-86-TEX-N-2E",[171,7360],{"id":7361,"d":7126},"MJX-86-TEX-N-35",[208,7363,7364],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,7365,7366,7371,7376],{"dataMmlNode":215},[208,7367,7368],{"dataMmlNode":226},[220,7369],{"dataC":694,"xLinkHref":7370},"#MJX-86-TEX-I-1D466",[208,7372,7373],{"dataMmlNode":218,"transform":7213},[220,7374],{"dataC":1089,"xLinkHref":7375},"#MJX-86-TEX-N-3D",[208,7377,7378,7382,7385],{"dataMmlNode":1103,"transform":7219},[220,7379],{"dataC":7380,"xLinkHref":7381},"33","#MJX-86-TEX-N-33",[220,7383],{"dataC":7164,"xLinkHref":7384,"transform":3275},"#MJX-86-TEX-N-2E",[220,7386],{"dataC":7168,"xLinkHref":7387,"transform":6434},"#MJX-86-TEX-N-35"," or ",[151,7390,7392],{"className":7391,"jax":155},[154],[157,7393,7396,7416],{"style":665,"xmlns":160,"width":7394,"height":7189,"role":163,"focusable":164,"viewBox":7395,"xmlnsXLink":166},"8.777ex","0 -666 3879.6 871",[168,7397,7398,7401,7404,7407,7410,7413],{},[171,7399],{"id":7400,"d":674},"MJX-87-TEX-I-1D466",[171,7402],{"id":7403,"d":1039},"MJX-87-TEX-N-3D",[171,7405],{"id":7406,"d":182},"MJX-87-TEX-N-2212",[171,7408],{"id":7409,"d":1055},"MJX-87-TEX-N-31",[171,7411],{"id":7412,"d":7122},"MJX-87-TEX-N-2E",[171,7414],{"id":7415,"d":1059},"MJX-87-TEX-N-32",[208,7417,7418],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,7419,7420,7425,7430,7435],{"dataMmlNode":215},[208,7421,7422],{"dataMmlNode":226},[220,7423],{"dataC":694,"xLinkHref":7424},"#MJX-87-TEX-I-1D466",[208,7426,7427],{"dataMmlNode":218,"transform":7213},[220,7428],{"dataC":1089,"xLinkHref":7429},"#MJX-87-TEX-N-3D",[208,7431,7432],{"dataMmlNode":218,"transform":7219},[220,7433],{"dataC":237,"xLinkHref":7434},"#MJX-87-TEX-N-2212",[208,7436,7438,7441,7444],{"dataMmlNode":1103,"transform":7437},"translate(2601.6,0)",[220,7439],{"dataC":1127,"xLinkHref":7440},"#MJX-87-TEX-N-31",[220,7442],{"dataC":7164,"xLinkHref":7443,"transform":3275},"#MJX-87-TEX-N-2E",[220,7445],{"dataC":1157,"xLinkHref":7446,"transform":6434},"#MJX-87-TEX-N-32",". What does a \"3.5\" or \"-1.2\" mean when the answer can only be 0 or 1? It's nonsense.",[28,7449,7451],{"id":7450},"the-solution-the-sigmoid-function","The Solution: The Sigmoid Function",[16,7453,7454,7455,7474,7475,7494],{},"Instead of a straight line, we want a curve that flatlines at ",[151,7456,7458],{"className":7457,"jax":155},[154],[157,7459,7460,7465],{"style":3975,"xmlns":160,"width":3337,"height":3976,"role":163,"focusable":164,"viewBox":3977,"xmlnsXLink":166},[168,7461,7462],{},[171,7463],{"id":7464,"d":1047},"MJX-88-TEX-N-30",[208,7466,7467],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,7468,7469],{"dataMmlNode":215},[208,7470,7471],{"dataMmlNode":1103},[220,7472],{"dataC":1107,"xLinkHref":7473},"#MJX-88-TEX-N-30"," and flatlines at ",[151,7476,7478],{"className":7477,"jax":155},[154],[157,7479,7480,7485],{"style":3336,"xmlns":160,"width":3337,"height":3338,"role":163,"focusable":164,"viewBox":3339,"xmlnsXLink":166},[168,7481,7482],{},[171,7483],{"id":7484,"d":1055},"MJX-89-TEX-N-31",[208,7486,7487],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,7488,7489],{"dataMmlNode":215},[208,7490,7491],{"dataMmlNode":1103},[220,7492],{"dataC":1127,"xLinkHref":7493},"#MJX-89-TEX-N-31",". We want the \"S-shape\" curve.",[16,7496,7497],{},[578,7498,7499,7503],{},[163,7500],{"alt":7501,"src":7502},"0.78","https:\u002F\u002Fraw.githubusercontent.com\u002FChinHongTan\u002Fblog\u002Fmain\u002Fpublic\u002Fimages\u002Fuploads\u002F1777302135699-Screenshot_2026-04-27_at_11.02.04_PM.png",[585,7504,7505],{},"Sigmoid Function Graph",[16,7507,7508,7509],{},"This is what the Sigmoid function does:\n",[151,7510,7512],{"className":7511,"jax":155},[154],[157,7513,7518,7553],{"style":7514,"xmlns":160,"width":7515,"height":7516,"role":163,"focusable":164,"viewBox":7517,"xmlnsXLink":166},"vertical-align: -1.324ex;","15.179ex","3.281ex","0 -864.9 6709.1 1450.3",[168,7519,7520,7523,7526,7529,7532,7535,7538,7541,7544,7547,7550],{},[171,7521],{"id":7522,"d":858},"MJX-90-TEX-I-210E",[171,7524],{"id":7525,"d":1043},"MJX-90-TEX-I-1D703",[171,7527],{"id":7528,"d":174},"MJX-90-TEX-N-28",[171,7530],{"id":7531,"d":178},"MJX-90-TEX-I-1D465",[171,7533],{"id":7534,"d":198},"MJX-90-TEX-N-29",[171,7536],{"id":7537,"d":1039},"MJX-90-TEX-N-3D",[171,7539],{"id":7540,"d":1055},"MJX-90-TEX-N-31",[171,7542],{"id":7543,"d":1051},"MJX-90-TEX-N-2B",[171,7545],{"id":7546,"d":354},"MJX-90-TEX-I-1D452",[171,7548],{"id":7549,"d":182},"MJX-90-TEX-N-2212",[171,7551],{"id":7552,"d":6987},"MJX-90-TEX-I-1D447",[208,7554,7555],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,7556,7557,7569,7574,7579,7584,7589],{"dataMmlNode":215},[208,7558,7559,7564],{"dataMmlNode":1093},[208,7560,7561],{"dataMmlNode":226},[220,7562],{"dataC":876,"xLinkHref":7563},"#MJX-90-TEX-I-210E",[208,7565,7566],{"dataMmlNode":226,"transform":7001},[220,7567],{"dataC":1099,"xLinkHref":7568},"#MJX-90-TEX-I-1D703",[208,7570,7571],{"dataMmlNode":218,"transform":7007},[220,7572],{"dataC":222,"xLinkHref":7573},"#MJX-90-TEX-N-28",[208,7575,7576],{"dataMmlNode":226,"transform":7013},[220,7577],{"dataC":230,"xLinkHref":7578},"#MJX-90-TEX-I-1D465",[208,7580,7581],{"dataMmlNode":218,"transform":7019},[220,7582],{"dataC":265,"xLinkHref":7583},"#MJX-90-TEX-N-29",[208,7585,7586],{"dataMmlNode":218,"transform":7025},[220,7587],{"dataC":1089,"xLinkHref":7588},"#MJX-90-TEX-N-3D",[208,7590,7591,7597,7640],{"dataMmlNode":1324,"transform":7031},[208,7592,7594],{"dataMmlNode":1103,"transform":7593},"translate(1340.7,394) scale(0.707)",[220,7595],{"dataC":1127,"xLinkHref":7596},"#MJX-90-TEX-N-31",[208,7598,7600,7604,7609],{"dataMmlNode":1821,"transform":7599},"translate(220,-527.4) scale(0.707)",[208,7601,7602],{"dataMmlNode":1103},[220,7603],{"dataC":1127,"xLinkHref":7596},[208,7605,7606],{"dataMmlNode":218,"transform":3275},[220,7607],{"dataC":1114,"xLinkHref":7608},"#MJX-90-TEX-N-2B",[208,7610,7611,7616],{"dataMmlNode":630,"transform":3630},[208,7612,7613],{"dataMmlNode":226},[220,7614],{"dataC":415,"xLinkHref":7615},"#MJX-90-TEX-I-1D452",[208,7617,7618,7623,7635],{"dataMmlNode":269,"transform":7061,"dataMjxTexclass":270},[208,7619,7620],{"dataMmlNode":218},[220,7621],{"dataC":237,"xLinkHref":7622},"#MJX-90-TEX-N-2212",[208,7624,7625,7629],{"dataMmlNode":630,"transform":6434},[208,7626,7627],{"dataMmlNode":226},[220,7628],{"dataC":1099,"xLinkHref":7568},[208,7630,7632],{"dataMmlNode":226,"transform":7631},"translate(502,289)",[220,7633],{"dataC":7078,"xLinkHref":7634},"#MJX-90-TEX-I-1D447",[208,7636,7638],{"dataMmlNode":226,"transform":7637},"translate(2034,0)",[220,7639],{"dataC":230,"xLinkHref":7578},[1339,7641],{"width":7642,"height":1342,"x":1343,"y":1344},2794.9,[16,7644,7645,7646,7688,7689,7692,7693,4771,7724,7279],{},"Think of ",[151,7647,7649],{"className":7648,"jax":155},[154],[157,7650,7654,7665],{"style":606,"xmlns":160,"width":7651,"height":7652,"role":163,"focusable":164,"viewBox":7653,"xmlnsXLink":166},"3.669ex","1.929ex","0 -841.7 1621.8 852.7",[168,7655,7656,7659,7662],{},[171,7657],{"id":7658,"d":1043},"MJX-91-TEX-I-1D703",[171,7660],{"id":7661,"d":6987},"MJX-91-TEX-I-1D447",[171,7663],{"id":7664,"d":178},"MJX-91-TEX-I-1D465",[208,7666,7667],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,7668,7669,7682],{"dataMmlNode":215},[208,7670,7671,7676],{"dataMmlNode":630},[208,7672,7673],{"dataMmlNode":226},[220,7674],{"dataC":1099,"xLinkHref":7675},"#MJX-91-TEX-I-1D703",[208,7677,7679],{"dataMmlNode":226,"transform":7678},"translate(502,363) scale(0.707)",[220,7680],{"dataC":7078,"xLinkHref":7681},"#MJX-91-TEX-I-1D447",[208,7683,7685],{"dataMmlNode":226,"transform":7684},"translate(1049.8,0)",[220,7686],{"dataC":230,"xLinkHref":7687},"#MJX-91-TEX-I-1D465"," as a ",[20,7690,7691],{},"\"raw confidence score\""," (which can be any number from ",[151,7694,7696],{"className":7695,"jax":155},[154],[157,7697,7700,7709],{"style":1888,"xmlns":160,"width":7698,"height":1890,"role":163,"focusable":164,"viewBox":7699,"xmlnsXLink":166},"4.023ex","0 -583 1778 665",[168,7701,7702,7705],{},[171,7703],{"id":7704,"d":182},"MJX-92-TEX-N-2212",[171,7706],{"id":7707,"d":7708},"MJX-92-TEX-N-221E","M55 217Q55 305 111 373T254 442Q342 442 419 381Q457 350 493 303L507 284L514 294Q618 442 747 442Q833 442 888 374T944 214Q944 128 889 59T743 -11Q657 -11 580 50Q542 81 506 128L492 147L485 137Q381 -11 252 -11Q166 -11 111 57T55 217ZM907 217Q907 285 869 341T761 397Q740 397 720 392T682 378T648 359T619 335T594 310T574 285T559 263T548 246L543 238L574 198Q605 158 622 138T664 94T714 61T765 51Q827 51 867 100T907 217ZM92 214Q92 145 131 89T239 33Q357 33 456 193L425 233Q364 312 334 337Q285 380 233 380Q171 380 132 331T92 214Z",[208,7710,7711],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,7712,7713,7718],{"dataMmlNode":215},[208,7714,7715],{"dataMmlNode":218},[220,7716],{"dataC":237,"xLinkHref":7717},"#MJX-92-TEX-N-2212",[208,7719,7720],{"dataMmlNode":226,"transform":6434},[220,7721],{"dataC":7722,"xLinkHref":7723},"221E","#MJX-92-TEX-N-221E",[151,7725,7727],{"className":7726,"jax":155},[154],[157,7728,7729,7737],{"style":1888,"xmlns":160,"width":7698,"height":1890,"role":163,"focusable":164,"viewBox":7699,"xmlnsXLink":166},[168,7730,7731,7734],{},[171,7732],{"id":7733,"d":1051},"MJX-93-TEX-N-2B",[171,7735],{"id":7736,"d":7708},"MJX-93-TEX-N-221E",[208,7738,7739],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,7740,7741,7746],{"dataMmlNode":215},[208,7742,7743],{"dataMmlNode":218},[220,7744],{"dataC":1114,"xLinkHref":7745},"#MJX-93-TEX-N-2B",[208,7747,7748],{"dataMmlNode":226,"transform":6434},[220,7749],{"dataC":7722,"xLinkHref":7750},"#MJX-93-TEX-N-221E",[520,7752,7753,7808,7886],{},[523,7754,7755,7756,7775,7776,7807],{},"If the raw score is ",[151,7757,7759],{"className":7758,"jax":155},[154],[157,7760,7761,7766],{"style":3975,"xmlns":160,"width":3337,"height":3976,"role":163,"focusable":164,"viewBox":3977,"xmlnsXLink":166},[168,7762,7763],{},[171,7764],{"id":7765,"d":1047},"MJX-94-TEX-N-30",[208,7767,7768],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,7769,7770],{"dataMmlNode":215},[208,7771,7772],{"dataMmlNode":1103},[220,7773],{"dataC":1107,"xLinkHref":7774},"#MJX-94-TEX-N-30",", the sigmoid turns it into exactly ",[151,7777,7779],{"className":7778,"jax":155},[154],[157,7780,7781,7792],{"style":3975,"xmlns":160,"width":7301,"height":3976,"role":163,"focusable":164,"viewBox":7302,"xmlnsXLink":166},[168,7782,7783,7786,7789],{},[171,7784],{"id":7785,"d":1047},"MJX-95-TEX-N-30",[171,7787],{"id":7788,"d":7122},"MJX-95-TEX-N-2E",[171,7790],{"id":7791,"d":7126},"MJX-95-TEX-N-35",[208,7793,7794],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,7795,7796],{"dataMmlNode":215},[208,7797,7798,7801,7804],{"dataMmlNode":1103},[220,7799],{"dataC":1107,"xLinkHref":7800},"#MJX-95-TEX-N-30",[220,7802],{"dataC":7164,"xLinkHref":7803,"transform":3275},"#MJX-95-TEX-N-2E",[220,7805],{"dataC":7168,"xLinkHref":7806,"transform":6434},"#MJX-95-TEX-N-35"," (50% probability).",[523,7809,7810,7811,7841,7842,7885],{},"If the raw score is a huge positive number (like ",[151,7812,7814],{"className":7813,"jax":155},[154],[157,7815,7818,7826],{"style":3975,"xmlns":160,"width":7816,"height":3976,"role":163,"focusable":164,"viewBox":7817,"xmlnsXLink":166},"3.394ex","0 -666 1500 688",[168,7819,7820,7823],{},[171,7821],{"id":7822,"d":1055},"MJX-96-TEX-N-31",[171,7824],{"id":7825,"d":1047},"MJX-96-TEX-N-30",[208,7827,7828],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,7829,7830],{"dataMmlNode":215},[208,7831,7832,7835,7838],{"dataMmlNode":1103},[220,7833],{"dataC":1127,"xLinkHref":7834},"#MJX-96-TEX-N-31",[220,7836],{"dataC":1107,"xLinkHref":7837,"transform":3275},"#MJX-96-TEX-N-30",[220,7839],{"dataC":1107,"xLinkHref":7837,"transform":7840},"translate(1000,0)","), the sigmoid squashes it to ",[151,7843,7845],{"className":7844,"jax":155},[154],[157,7846,7849,7861],{"style":3975,"xmlns":160,"width":7847,"height":3976,"role":163,"focusable":164,"viewBox":7848,"xmlnsXLink":166},"6.285ex","0 -666 2778 688",[168,7850,7851,7854,7857],{},[171,7852],{"id":7853,"d":1047},"MJX-97-TEX-N-30",[171,7855],{"id":7856,"d":7122},"MJX-97-TEX-N-2E",[171,7858],{"id":7859,"d":7860},"MJX-97-TEX-N-39","M352 287Q304 211 232 211Q154 211 104 270T44 396Q42 412 42 436V444Q42 537 111 606Q171 666 243 666Q245 666 249 666T257 665H261Q273 665 286 663T323 651T370 619T413 560Q456 472 456 334Q456 194 396 97Q361 41 312 10T208 -22Q147 -22 108 7T68 93T121 149Q143 149 158 135T173 96Q173 78 164 65T148 49T135 44L131 43Q131 41 138 37T164 27T206 22H212Q272 22 313 86Q352 142 352 280V287ZM244 248Q292 248 321 297T351 430Q351 508 343 542Q341 552 337 562T323 588T293 615T246 625Q208 625 181 598Q160 576 154 546T147 441Q147 358 152 329T172 282Q197 248 244 248Z",[208,7862,7863],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,7864,7865],{"dataMmlNode":215},[208,7866,7867,7870,7873,7877,7879,7882],{"dataMmlNode":1103},[220,7868],{"dataC":1107,"xLinkHref":7869},"#MJX-97-TEX-N-30",[220,7871],{"dataC":7164,"xLinkHref":7872,"transform":3275},"#MJX-97-TEX-N-2E",[220,7874],{"dataC":7875,"xLinkHref":7876,"transform":6434},"39","#MJX-97-TEX-N-39",[220,7878],{"dataC":7875,"xLinkHref":7876,"transform":3630},[220,7880],{"dataC":7875,"xLinkHref":7876,"transform":7881},"translate(1778,0)",[220,7883],{"dataC":7875,"xLinkHref":7876,"transform":7884},"translate(2278,0)"," (99.99% probability it is Malignant).",[523,7887,7888,7889,7841,7927,7964],{},"If the raw score is a huge negative number (like ",[151,7890,7892],{"className":7891,"jax":155},[154],[157,7893,7897,7908],{"style":1888,"xmlns":160,"width":7894,"height":7895,"role":163,"focusable":164,"viewBox":7896,"xmlnsXLink":166},"5.154ex","1.692ex","0 -666 2278 748",[168,7898,7899,7902,7905],{},[171,7900],{"id":7901,"d":182},"MJX-98-TEX-N-2212",[171,7903],{"id":7904,"d":1055},"MJX-98-TEX-N-31",[171,7906],{"id":7907,"d":1047},"MJX-98-TEX-N-30",[208,7909,7910],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,7911,7912,7917],{"dataMmlNode":215},[208,7913,7914],{"dataMmlNode":218},[220,7915],{"dataC":237,"xLinkHref":7916},"#MJX-98-TEX-N-2212",[208,7918,7919,7922,7925],{"dataMmlNode":1103,"transform":6434},[220,7920],{"dataC":1127,"xLinkHref":7921},"#MJX-98-TEX-N-31",[220,7923],{"dataC":1107,"xLinkHref":7924,"transform":3275},"#MJX-98-TEX-N-30",[220,7926],{"dataC":1107,"xLinkHref":7924,"transform":7840},[151,7928,7930],{"className":7929,"jax":155},[154],[157,7931,7932,7943],{"style":3975,"xmlns":160,"width":7847,"height":3976,"role":163,"focusable":164,"viewBox":7848,"xmlnsXLink":166},[168,7933,7934,7937,7940],{},[171,7935],{"id":7936,"d":1047},"MJX-99-TEX-N-30",[171,7938],{"id":7939,"d":7122},"MJX-99-TEX-N-2E",[171,7941],{"id":7942,"d":1055},"MJX-99-TEX-N-31",[208,7944,7945],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,7946,7947],{"dataMmlNode":215},[208,7948,7949,7952,7955,7957,7959,7961],{"dataMmlNode":1103},[220,7950],{"dataC":1107,"xLinkHref":7951},"#MJX-99-TEX-N-30",[220,7953],{"dataC":7164,"xLinkHref":7954,"transform":3275},"#MJX-99-TEX-N-2E",[220,7956],{"dataC":1107,"xLinkHref":7951,"transform":6434},[220,7958],{"dataC":1107,"xLinkHref":7951,"transform":3630},[220,7960],{"dataC":1107,"xLinkHref":7951,"transform":7881},[220,7962],{"dataC":1127,"xLinkHref":7963,"transform":7884},"#MJX-99-TEX-N-31"," (0.01% probability it is Malignant).",[16,7966,7967],{},[20,7968,7969],{},"Logistic Regression is just Linear Regression wrapped in a translator that turns \"raw scores\" into \"probabilities.\"",[28,7971,7973],{"id":7972},"fitting-logistic-regression","Fitting Logistic Regression",[16,7975,7976,7977,7980],{},"Instead of minimizing squared error, we use ",[20,7978,7979],{},"maximum likelihood estimation",". We assume:",[520,7982,7983,8122],{},[523,7984,7985],{},[151,7986,7988],{"className":7987,"jax":155},[154],[157,7989,7992,8030],{"style":159,"xmlns":160,"width":7990,"height":162,"role":163,"focusable":164,"viewBox":7991,"xmlnsXLink":166},"20.081ex","0 -750 8875.8 1000",[168,7993,7994,7998,8001,8004,8007,8010,8014,8017,8021,8024,8027],{},[171,7995],{"id":7996,"d":7997},"MJX-100-TEX-I-1D443","M287 628Q287 635 230 637Q206 637 199 638T192 648Q192 649 194 659Q200 679 203 681T397 683Q587 682 600 680Q664 669 707 631T751 530Q751 453 685 389Q616 321 507 303Q500 302 402 301H307L277 182Q247 66 247 59Q247 55 248 54T255 50T272 48T305 46H336Q342 37 342 35Q342 19 335 5Q330 0 319 0Q316 0 282 1T182 2Q120 2 87 2T51 1Q33 1 33 11Q33 13 36 25Q40 41 44 43T67 46Q94 46 127 49Q141 52 146 61Q149 65 218 339T287 628ZM645 554Q645 567 643 575T634 597T609 619T560 635Q553 636 480 637Q463 637 445 637T416 636T404 636Q391 635 386 627Q384 621 367 550T332 412T314 344Q314 342 395 342H407H430Q542 342 590 392Q617 419 631 471T645 554Z",[171,7999],{"id":8000,"d":174},"MJX-100-TEX-N-28",[171,8002],{"id":8003,"d":674},"MJX-100-TEX-I-1D466",[171,8005],{"id":8006,"d":1039},"MJX-100-TEX-N-3D",[171,8008],{"id":8009,"d":1055},"MJX-100-TEX-N-31",[171,8011],{"id":8012,"d":8013},"MJX-100-TEX-N-7C","M139 -249H137Q125 -249 119 -235V251L120 737Q130 750 139 750Q152 750 159 735V-235Q151 -249 141 -249H139Z",[171,8015],{"id":8016,"d":178},"MJX-100-TEX-I-1D465",[171,8018],{"id":8019,"d":8020},"MJX-100-TEX-N-3B","M78 370Q78 394 95 412T138 430Q162 430 180 414T199 371Q199 346 182 328T139 310T96 327T78 370ZM78 60Q78 85 94 103T137 121Q202 121 202 8Q202 -44 183 -94T144 -169T118 -194Q115 -194 106 -186T95 -174Q94 -171 107 -155T137 -107T160 -38Q161 -32 162 -22T165 -4T165 4Q165 5 161 4T142 0Q110 0 94 18T78 60Z",[171,8022],{"id":8023,"d":1043},"MJX-100-TEX-I-1D703",[171,8025],{"id":8026,"d":198},"MJX-100-TEX-N-29",[171,8028],{"id":8029,"d":858},"MJX-100-TEX-I-210E",[208,8031,8032],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,8033,8034,8040,8046,8052,8058,8064,8071,8077,8084,8090,8096,8101,8107,8112,8117],{"dataMmlNode":215},[208,8035,8036],{"dataMmlNode":226},[220,8037],{"dataC":8038,"xLinkHref":8039},"1D443","#MJX-100-TEX-I-1D443",[208,8041,8043],{"dataMmlNode":218,"transform":8042},"translate(751,0)",[220,8044],{"dataC":222,"xLinkHref":8045},"#MJX-100-TEX-N-28",[208,8047,8049],{"dataMmlNode":226,"transform":8048},"translate(1140,0)",[220,8050],{"dataC":694,"xLinkHref":8051},"#MJX-100-TEX-I-1D466",[208,8053,8055],{"dataMmlNode":218,"transform":8054},"translate(1907.8,0)",[220,8056],{"dataC":1089,"xLinkHref":8057},"#MJX-100-TEX-N-3D",[208,8059,8061],{"dataMmlNode":1103,"transform":8060},"translate(2963.6,0)",[220,8062],{"dataC":1127,"xLinkHref":8063},"#MJX-100-TEX-N-31",[208,8065,8067],{"dataMmlNode":218,"transform":8066},"translate(3463.6,0) translate(0 -0.5)",[220,8068],{"dataC":8069,"xLinkHref":8070},"7C","#MJX-100-TEX-N-7C",[208,8072,8074],{"dataMmlNode":226,"transform":8073},"translate(3741.6,0)",[220,8075],{"dataC":230,"xLinkHref":8076},"#MJX-100-TEX-I-1D465",[208,8078,8080],{"dataMmlNode":218,"transform":8079},"translate(4313.6,0)",[220,8081],{"dataC":8082,"xLinkHref":8083},"3B","#MJX-100-TEX-N-3B",[208,8085,8087],{"dataMmlNode":226,"transform":8086},"translate(4758.2,0)",[220,8088],{"dataC":1099,"xLinkHref":8089},"#MJX-100-TEX-I-1D703",[208,8091,8093],{"dataMmlNode":218,"transform":8092},"translate(5227.2,0)",[220,8094],{"dataC":265,"xLinkHref":8095},"#MJX-100-TEX-N-29",[208,8097,8099],{"dataMmlNode":218,"transform":8098},"translate(5894,0)",[220,8100],{"dataC":1089,"xLinkHref":8057},[208,8102,8104],{"dataMmlNode":226,"transform":8103},"translate(6949.8,0)",[220,8105],{"dataC":876,"xLinkHref":8106},"#MJX-100-TEX-I-210E",[208,8108,8110],{"dataMmlNode":218,"transform":8109},"translate(7525.8,0)",[220,8111],{"dataC":222,"xLinkHref":8045},[208,8113,8115],{"dataMmlNode":226,"transform":8114},"translate(7914.8,0)",[220,8116],{"dataC":230,"xLinkHref":8076},[208,8118,8120],{"dataMmlNode":218,"transform":8119},"translate(8486.8,0)",[220,8121],{"dataC":265,"xLinkHref":8095},[523,8123,8124],{},[151,8125,8127],{"className":8126,"jax":155},[154],[157,8128,8131,8172],{"style":159,"xmlns":160,"width":8129,"height":162,"role":163,"focusable":164,"viewBox":8130,"xmlnsXLink":166},"23.978ex","0 -750 10598.2 1000",[168,8132,8133,8136,8139,8142,8145,8148,8151,8154,8157,8160,8163,8166,8169],{},[171,8134],{"id":8135,"d":7997},"MJX-101-TEX-I-1D443",[171,8137],{"id":8138,"d":174},"MJX-101-TEX-N-28",[171,8140],{"id":8141,"d":674},"MJX-101-TEX-I-1D466",[171,8143],{"id":8144,"d":1039},"MJX-101-TEX-N-3D",[171,8146],{"id":8147,"d":1047},"MJX-101-TEX-N-30",[171,8149],{"id":8150,"d":8013},"MJX-101-TEX-N-7C",[171,8152],{"id":8153,"d":178},"MJX-101-TEX-I-1D465",[171,8155],{"id":8156,"d":8020},"MJX-101-TEX-N-3B",[171,8158],{"id":8159,"d":1043},"MJX-101-TEX-I-1D703",[171,8161],{"id":8162,"d":198},"MJX-101-TEX-N-29",[171,8164],{"id":8165,"d":1055},"MJX-101-TEX-N-31",[171,8167],{"id":8168,"d":182},"MJX-101-TEX-N-2212",[171,8170],{"id":8171,"d":858},"MJX-101-TEX-I-210E",[208,8173,8174],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,8175,8176,8181,8186,8191,8196,8201,8206,8211,8216,8221,8226,8230,8235,8241,8247,8252,8257],{"dataMmlNode":215},[208,8177,8178],{"dataMmlNode":226},[220,8179],{"dataC":8038,"xLinkHref":8180},"#MJX-101-TEX-I-1D443",[208,8182,8183],{"dataMmlNode":218,"transform":8042},[220,8184],{"dataC":222,"xLinkHref":8185},"#MJX-101-TEX-N-28",[208,8187,8188],{"dataMmlNode":226,"transform":8048},[220,8189],{"dataC":694,"xLinkHref":8190},"#MJX-101-TEX-I-1D466",[208,8192,8193],{"dataMmlNode":218,"transform":8054},[220,8194],{"dataC":1089,"xLinkHref":8195},"#MJX-101-TEX-N-3D",[208,8197,8198],{"dataMmlNode":1103,"transform":8060},[220,8199],{"dataC":1107,"xLinkHref":8200},"#MJX-101-TEX-N-30",[208,8202,8203],{"dataMmlNode":218,"transform":8066},[220,8204],{"dataC":8069,"xLinkHref":8205},"#MJX-101-TEX-N-7C",[208,8207,8208],{"dataMmlNode":226,"transform":8073},[220,8209],{"dataC":230,"xLinkHref":8210},"#MJX-101-TEX-I-1D465",[208,8212,8213],{"dataMmlNode":218,"transform":8079},[220,8214],{"dataC":8082,"xLinkHref":8215},"#MJX-101-TEX-N-3B",[208,8217,8218],{"dataMmlNode":226,"transform":8086},[220,8219],{"dataC":1099,"xLinkHref":8220},"#MJX-101-TEX-I-1D703",[208,8222,8223],{"dataMmlNode":218,"transform":8092},[220,8224],{"dataC":265,"xLinkHref":8225},"#MJX-101-TEX-N-29",[208,8227,8228],{"dataMmlNode":218,"transform":8098},[220,8229],{"dataC":1089,"xLinkHref":8195},[208,8231,8232],{"dataMmlNode":1103,"transform":8103},[220,8233],{"dataC":1127,"xLinkHref":8234},"#MJX-101-TEX-N-31",[208,8236,8238],{"dataMmlNode":218,"transform":8237},"translate(7672,0)",[220,8239],{"dataC":237,"xLinkHref":8240},"#MJX-101-TEX-N-2212",[208,8242,8244],{"dataMmlNode":226,"transform":8243},"translate(8672.2,0)",[220,8245],{"dataC":876,"xLinkHref":8246},"#MJX-101-TEX-I-210E",[208,8248,8250],{"dataMmlNode":218,"transform":8249},"translate(9248.2,0)",[220,8251],{"dataC":222,"xLinkHref":8185},[208,8253,8255],{"dataMmlNode":226,"transform":8254},"translate(9637.2,0)",[220,8256],{"dataC":230,"xLinkHref":8210},[208,8258,8260],{"dataMmlNode":218,"transform":8259},"translate(10209.2,0)",[220,8261],{"dataC":265,"xLinkHref":8225},[16,8263,8264],{},"The log-likelihood is:",[151,8266,8268],{"className":8267,"jax":155,"display":1017},[154],[157,8269,8272,8333],{"style":1241,"xmlns":160,"width":8270,"height":4460,"role":163,"focusable":164,"viewBox":8271,"xmlnsXLink":166},"52.425ex","0 -1149.5 23171.9 2341.7",[168,8273,8274,8278,8281,8284,8287,8290,8293,8296,8299,8302,8305,8309,8312,8315,8318,8321,8324,8327,8330],{},[171,8275],{"id":8276,"d":8277},"MJX-102-TEX-I-2113","M345 104T349 104T361 95T369 80T352 59Q268 -20 206 -20Q170 -20 146 3T113 53T99 104L94 129Q94 130 79 116T48 86T28 70Q22 70 15 79T7 94Q7 98 12 103T58 147L91 179V185Q91 186 91 191T92 200Q92 282 128 400T223 612T336 705Q397 705 397 636V627Q397 453 194 233Q185 223 180 218T174 211T171 208T165 201L163 186Q159 142 159 123Q159 17 208 17Q228 17 253 30T293 56T335 94Q345 104 349 104ZM360 634Q360 655 354 661T336 668Q328 668 322 666T302 645T272 592Q252 547 229 467T192 330L179 273Q179 272 186 280T204 300T221 322Q327 453 355 590Q360 612 360 634Z",[171,8279],{"id":8280,"d":174},"MJX-102-TEX-N-28",[171,8282],{"id":8283,"d":1043},"MJX-102-TEX-I-1D703",[171,8285],{"id":8286,"d":198},"MJX-102-TEX-N-29",[171,8288],{"id":8289,"d":1039},"MJX-102-TEX-N-3D",[171,8291],{"id":8292,"d":1272},"MJX-102-TEX-LO-2211",[171,8294],{"id":8295,"d":190},"MJX-102-TEX-I-1D456",[171,8297],{"id":8298,"d":4233},"MJX-102-TEX-LO-5B",[171,8300],{"id":8301,"d":674},"MJX-102-TEX-I-1D466",[171,8303],{"id":8304,"d":6397},"MJX-102-TEX-N-6C",[171,8306],{"id":8307,"d":8308},"MJX-102-TEX-N-6F","M28 214Q28 309 93 378T250 448Q340 448 405 380T471 215Q471 120 407 55T250 -10Q153 -10 91 57T28 214ZM250 30Q372 30 372 193V225V250Q372 272 371 288T364 326T348 362T317 390T268 410Q263 411 252 411Q222 411 195 399Q152 377 139 338T126 246V226Q126 130 145 91Q177 30 250 30Z",[171,8310],{"id":8311,"d":6409},"MJX-102-TEX-N-67",[171,8313],{"id":8314,"d":5496},"MJX-102-TEX-N-2061",[171,8316],{"id":8317,"d":858},"MJX-102-TEX-I-210E",[171,8319],{"id":8320,"d":178},"MJX-102-TEX-I-1D465",[171,8322],{"id":8323,"d":1051},"MJX-102-TEX-N-2B",[171,8325],{"id":8326,"d":1055},"MJX-102-TEX-N-31",[171,8328],{"id":8329,"d":182},"MJX-102-TEX-N-2212",[171,8331],{"id":8332,"d":4258},"MJX-102-TEX-LO-5D",[208,8334,8335],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,8336,8337,8343,8349,8355,8361,8367,8380],{"dataMmlNode":215},[208,8338,8339],{"dataMmlNode":226},[220,8340],{"dataC":8341,"xLinkHref":8342},"2113","#MJX-102-TEX-I-2113",[208,8344,8346],{"dataMmlNode":218,"transform":8345},"translate(417,0)",[220,8347],{"dataC":222,"xLinkHref":8348},"#MJX-102-TEX-N-28",[208,8350,8352],{"dataMmlNode":226,"transform":8351},"translate(806,0)",[220,8353],{"dataC":1099,"xLinkHref":8354},"#MJX-102-TEX-I-1D703",[208,8356,8358],{"dataMmlNode":218,"transform":8357},"translate(1275,0)",[220,8359],{"dataC":265,"xLinkHref":8360},"#MJX-102-TEX-N-29",[208,8362,8364],{"dataMmlNode":218,"transform":8363},"translate(1941.8,0)",[220,8365],{"dataC":1089,"xLinkHref":8366},"#MJX-102-TEX-N-3D",[208,8368,8370,8375],{"dataMmlNode":1347,"transform":8369},"translate(2997.6,0)",[208,8371,8372],{"dataMmlNode":218},[220,8373],{"dataC":1353,"xLinkHref":8374},"#MJX-102-TEX-LO-2211",[208,8376,8377],{"dataMmlNode":226,"transform":1357},[220,8378],{"dataC":251,"xLinkHref":8379},"#MJX-102-TEX-I-1D456",[208,8381,8383,8388,8409,8422,8428,8434,8439,8461,8466,8472,8477,8483,8489,8510,8515,8524,8529,8533,8538,8543,8548,8553,8574,8579,8584],{"dataMmlNode":1821,"transform":8382},"translate(4608.2,0)",[208,8384,8385],{"dataMmlNode":218,"transform":2463},[220,8386],{"dataC":4334,"xLinkHref":8387},"#MJX-102-TEX-LO-5B",[208,8389,8390,8395],{"dataMmlNode":630,"transform":4338},[208,8391,8392],{"dataMmlNode":226},[220,8393],{"dataC":694,"xLinkHref":8394},"#MJX-102-TEX-I-1D466",[208,8396,8397,8401,8405],{"dataMmlNode":269,"transform":1421,"dataMjxTexclass":270},[208,8398,8399],{"dataMmlNode":218},[220,8400],{"dataC":222,"xLinkHref":8348},[208,8402,8403],{"dataMmlNode":226,"transform":227},[220,8404],{"dataC":251,"xLinkHref":8379},[208,8406,8407],{"dataMmlNode":218,"transform":651},[220,8408],{"dataC":265,"xLinkHref":8360},[208,8410,8412,8415,8419],{"dataMmlNode":226,"transform":8411},"translate(2005.7,0)",[220,8413],{"dataC":6437,"xLinkHref":8414},"#MJX-102-TEX-N-6C",[220,8416],{"dataC":8417,"xLinkHref":8418,"transform":1788},"6F","#MJX-102-TEX-N-6F",[220,8420],{"dataC":6449,"xLinkHref":8421,"transform":6434},"#MJX-102-TEX-N-67",[208,8423,8425],{"dataMmlNode":218,"transform":8424},"translate(3283.7,0)",[220,8426],{"dataC":5574,"xLinkHref":8427},"#MJX-102-TEX-N-2061",[208,8429,8431],{"dataMmlNode":226,"transform":8430},"translate(3450.4,0)",[220,8432],{"dataC":876,"xLinkHref":8433},"#MJX-102-TEX-I-210E",[208,8435,8437],{"dataMmlNode":218,"transform":8436},"translate(4026.4,0)",[220,8438],{"dataC":222,"xLinkHref":8348},[208,8440,8442,8447],{"dataMmlNode":630,"transform":8441},"translate(4415.4,0)",[208,8443,8444],{"dataMmlNode":226},[220,8445],{"dataC":230,"xLinkHref":8446},"#MJX-102-TEX-I-1D465",[208,8448,8449,8453,8457],{"dataMmlNode":269,"transform":1387,"dataMjxTexclass":270},[208,8450,8451],{"dataMmlNode":218},[220,8452],{"dataC":222,"xLinkHref":8348},[208,8454,8455],{"dataMmlNode":226,"transform":227},[220,8456],{"dataC":251,"xLinkHref":8379},[208,8458,8459],{"dataMmlNode":218,"transform":651},[220,8460],{"dataC":265,"xLinkHref":8360},[208,8462,8464],{"dataMmlNode":218,"transform":8463},"translate(5864.5,0)",[220,8465],{"dataC":265,"xLinkHref":8360},[208,8467,8469],{"dataMmlNode":218,"transform":8468},"translate(6475.7,0)",[220,8470],{"dataC":1114,"xLinkHref":8471},"#MJX-102-TEX-N-2B",[208,8473,8475],{"dataMmlNode":218,"transform":8474},"translate(7475.9,0)",[220,8476],{"dataC":222,"xLinkHref":8348},[208,8478,8480],{"dataMmlNode":1103,"transform":8479},"translate(7864.9,0)",[220,8481],{"dataC":1127,"xLinkHref":8482},"#MJX-102-TEX-N-31",[208,8484,8486],{"dataMmlNode":218,"transform":8485},"translate(8587.2,0)",[220,8487],{"dataC":237,"xLinkHref":8488},"#MJX-102-TEX-N-2212",[208,8490,8492,8496],{"dataMmlNode":630,"transform":8491},"translate(9587.4,0)",[208,8493,8494],{"dataMmlNode":226},[220,8495],{"dataC":694,"xLinkHref":8394},[208,8497,8498,8502,8506],{"dataMmlNode":269,"transform":1421,"dataMjxTexclass":270},[208,8499,8500],{"dataMmlNode":218},[220,8501],{"dataC":222,"xLinkHref":8348},[208,8503,8504],{"dataMmlNode":226,"transform":227},[220,8505],{"dataC":251,"xLinkHref":8379},[208,8507,8508],{"dataMmlNode":218,"transform":651},[220,8509],{"dataC":265,"xLinkHref":8360},[208,8511,8513],{"dataMmlNode":218,"transform":8512},"translate(10954.5,0)",[220,8514],{"dataC":265,"xLinkHref":8360},[208,8516,8518,8520,8522],{"dataMmlNode":226,"transform":8517},"translate(11510.1,0)",[220,8519],{"dataC":6437,"xLinkHref":8414},[220,8521],{"dataC":8417,"xLinkHref":8418,"transform":1788},[220,8523],{"dataC":6449,"xLinkHref":8421,"transform":6434},[208,8525,8527],{"dataMmlNode":218,"transform":8526},"translate(12788.1,0)",[220,8528],{"dataC":5574,"xLinkHref":8427},[208,8530,8531],{"dataMmlNode":218,"transform":8526},[220,8532],{"dataC":222,"xLinkHref":8348},[208,8534,8536],{"dataMmlNode":1103,"transform":8535},"translate(13177.1,0)",[220,8537],{"dataC":1127,"xLinkHref":8482},[208,8539,8541],{"dataMmlNode":218,"transform":8540},"translate(13899.4,0)",[220,8542],{"dataC":237,"xLinkHref":8488},[208,8544,8546],{"dataMmlNode":226,"transform":8545},"translate(14899.6,0)",[220,8547],{"dataC":876,"xLinkHref":8433},[208,8549,8551],{"dataMmlNode":218,"transform":8550},"translate(15475.6,0)",[220,8552],{"dataC":222,"xLinkHref":8348},[208,8554,8556,8560],{"dataMmlNode":630,"transform":8555},"translate(15864.6,0)",[208,8557,8558],{"dataMmlNode":226},[220,8559],{"dataC":230,"xLinkHref":8446},[208,8561,8562,8566,8570],{"dataMmlNode":269,"transform":1387,"dataMjxTexclass":270},[208,8563,8564],{"dataMmlNode":218},[220,8565],{"dataC":222,"xLinkHref":8348},[208,8567,8568],{"dataMmlNode":226,"transform":227},[220,8569],{"dataC":251,"xLinkHref":8379},[208,8571,8572],{"dataMmlNode":218,"transform":651},[220,8573],{"dataC":265,"xLinkHref":8360},[208,8575,8577],{"dataMmlNode":218,"transform":8576},"translate(17313.7,0)",[220,8578],{"dataC":265,"xLinkHref":8360},[208,8580,8582],{"dataMmlNode":218,"transform":8581},"translate(17702.7,0)",[220,8583],{"dataC":265,"xLinkHref":8360},[208,8585,8587],{"dataMmlNode":218,"transform":8586},"translate(18091.7,0) translate(0 -0.5)",[220,8588],{"dataC":4445,"xLinkHref":8589},"#MJX-102-TEX-LO-5D",[16,8591,8592,8593,8596,8597,8600],{},"We ",[20,8594,8595],{},"maximize"," this using — you guessed it — ",[20,8598,8599],{},"gradient descent",". The resulting update rule looks remarkably similar to the linear regression one:",[151,8602,8604],{"className":8603,"jax":155,"display":1017},[154],[157,8605,8606,8647],{"style":1965,"xmlns":160,"width":1966,"height":1967,"role":163,"focusable":164,"viewBox":1968,"xmlnsXLink":166},[168,8607,8608,8611,8614,8617,8620,8623,8626,8629,8632,8635,8638,8641,8644],{},[171,8609],{"id":8610,"d":1043},"MJX-103-TEX-I-1D703",[171,8612],{"id":8613,"d":1732},"MJX-103-TEX-I-1D457",[171,8615],{"id":8616,"d":1736},"MJX-103-TEX-N-3A",[171,8618],{"id":8619,"d":1039},"MJX-103-TEX-N-3D",[171,8621],{"id":8622,"d":1051},"MJX-103-TEX-N-2B",[171,8624],{"id":8625,"d":1746},"MJX-103-TEX-I-1D6FC",[171,8627],{"id":8628,"d":174},"MJX-103-TEX-N-28",[171,8630],{"id":8631,"d":674},"MJX-103-TEX-I-1D466",[171,8633],{"id":8634,"d":190},"MJX-103-TEX-I-1D456",[171,8636],{"id":8637,"d":198},"MJX-103-TEX-N-29",[171,8639],{"id":8640,"d":182},"MJX-103-TEX-N-2212",[171,8642],{"id":8643,"d":858},"MJX-103-TEX-I-210E",[171,8645],{"id":8646,"d":178},"MJX-103-TEX-I-1D465",[208,8648,8649],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,8650,8651,8663,8675,8685,8690,8695,8700,8723,8728,8733,8737,8758,8762,8766],{"dataMmlNode":215},[208,8652,8653,8658],{"dataMmlNode":1093},[208,8654,8655],{"dataMmlNode":226},[220,8656],{"dataC":1099,"xLinkHref":8657},"#MJX-103-TEX-I-1D703",[208,8659,8660],{"dataMmlNode":226,"transform":1104},[220,8661],{"dataC":1775,"xLinkHref":8662},"#MJX-103-TEX-I-1D457",[208,8664,8665,8670],{"dataMmlNode":218,"transform":1779},[208,8666,8667],{"dataMmlNode":980},[220,8668],{"dataC":1784,"xLinkHref":8669},"#MJX-103-TEX-N-3A",[208,8671,8672],{"dataMmlNode":980,"transform":1788},[220,8673],{"dataC":1089,"xLinkHref":8674},"#MJX-103-TEX-N-3D",[208,8676,8677,8681],{"dataMmlNode":1093,"transform":1794},[208,8678,8679],{"dataMmlNode":226},[220,8680],{"dataC":1099,"xLinkHref":8657},[208,8682,8683],{"dataMmlNode":226,"transform":1104},[220,8684],{"dataC":1775,"xLinkHref":8662},[208,8686,8687],{"dataMmlNode":218,"transform":1805},[220,8688],{"dataC":1114,"xLinkHref":8689},"#MJX-103-TEX-N-2B",[208,8691,8692],{"dataMmlNode":226,"transform":1811},[220,8693],{"dataC":1814,"xLinkHref":8694},"#MJX-103-TEX-I-1D6FC",[208,8696,8697],{"dataMmlNode":218,"transform":1818},[220,8698],{"dataC":222,"xLinkHref":8699},"#MJX-103-TEX-N-28",[208,8701,8702,8707],{"dataMmlNode":630,"transform":2065},[208,8703,8704],{"dataMmlNode":226},[220,8705],{"dataC":694,"xLinkHref":8706},"#MJX-103-TEX-I-1D466",[208,8708,8709,8713,8718],{"dataMmlNode":269,"transform":1421,"dataMjxTexclass":270},[208,8710,8711],{"dataMmlNode":218},[220,8712],{"dataC":222,"xLinkHref":8699},[208,8714,8715],{"dataMmlNode":226,"transform":227},[220,8716],{"dataC":251,"xLinkHref":8717},"#MJX-103-TEX-I-1D456",[208,8719,8720],{"dataMmlNode":218,"transform":651},[220,8721],{"dataC":265,"xLinkHref":8722},"#MJX-103-TEX-N-29",[208,8724,8725],{"dataMmlNode":218,"transform":2089},[220,8726],{"dataC":237,"xLinkHref":8727},"#MJX-103-TEX-N-2212",[208,8729,8730],{"dataMmlNode":226,"transform":2095},[220,8731],{"dataC":876,"xLinkHref":8732},"#MJX-103-TEX-I-210E",[208,8734,8735],{"dataMmlNode":218,"transform":2101},[220,8736],{"dataC":222,"xLinkHref":8699},[208,8738,8739,8744],{"dataMmlNode":630,"transform":2106},[208,8740,8741],{"dataMmlNode":226},[220,8742],{"dataC":230,"xLinkHref":8743},"#MJX-103-TEX-I-1D465",[208,8745,8746,8750,8754],{"dataMmlNode":269,"transform":1387,"dataMjxTexclass":270},[208,8747,8748],{"dataMmlNode":218},[220,8749],{"dataC":222,"xLinkHref":8699},[208,8751,8752],{"dataMmlNode":226,"transform":227},[220,8753],{"dataC":251,"xLinkHref":8717},[208,8755,8756],{"dataMmlNode":218,"transform":651},[220,8757],{"dataC":265,"xLinkHref":8722},[208,8759,8760],{"dataMmlNode":218,"transform":2128},[220,8761],{"dataC":265,"xLinkHref":8722},[208,8763,8764],{"dataMmlNode":218,"transform":2133},[220,8765],{"dataC":265,"xLinkHref":8722},[208,8767,8768,8772,8786],{"dataMmlNode":2138,"transform":2139},[208,8769,8770],{"dataMmlNode":226},[220,8771],{"dataC":230,"xLinkHref":8743},[208,8773,8774,8778,8782],{"dataMmlNode":269,"transform":2146,"dataMjxTexclass":270},[208,8775,8776],{"dataMmlNode":218},[220,8777],{"dataC":222,"xLinkHref":8699},[208,8779,8780],{"dataMmlNode":226,"transform":227},[220,8781],{"dataC":251,"xLinkHref":8717},[208,8783,8784],{"dataMmlNode":218,"transform":651},[220,8785],{"dataC":265,"xLinkHref":8722},[208,8787,8788],{"dataMmlNode":226,"transform":2161},[220,8789],{"dataC":1775,"xLinkHref":8662},[16,8791,8792],{},"The form is identical, but remember that h(x) is now the sigmoid function, not a linear function. This elegant symmetry is one of the beautiful things about these algorithms.",[69,8794,8796,8799],{"className":8795},[72,73],[75,8797,8798],{},"The Log-Likelihood Cost Function",[79,8800,8802,8805,8808,8814,9070,9073,9605,9612,9616,9619,9622,9814,9819],{"className":8801},[82],[16,8803,8804],{},"In Linear Regression, we used the Least-Squares cost function (minimising the squared error).",[16,8806,8807],{},"If we try to use Least-Squares with our new curvy Sigmoid function, the math breaks. It creates a \"wavy\" 3D landscape with lots of fake valleys (local minima). Gradient descent gets stuck and can't find the bottom.",[16,8809,8810,8811,1542],{},"So, statisticians use a different way to measure error called ",[20,8812,8813],{},"Maximum Likelihood",[16,8815,8816],{},[151,8817,8819],{"className":8818,"jax":155},[154],[157,8820,8823,8882],{"style":159,"xmlns":160,"width":8821,"height":162,"role":163,"focusable":164,"viewBox":8822,"xmlnsXLink":166},"44.115ex","0 -750 19498.9 1000",[168,8824,8825,8828,8831,8834,8837,8840,8844,8848,8851,8854,8857,8860,8863,8866,8869,8872,8875,8878],{},[171,8826],{"id":8827,"d":8277},"MJX-104-TEX-I-2113",[171,8829],{"id":8830,"d":174},"MJX-104-TEX-N-28",[171,8832],{"id":8833,"d":1043},"MJX-104-TEX-I-1D703",[171,8835],{"id":8836,"d":198},"MJX-104-TEX-N-29",[171,8838],{"id":8839,"d":1039},"MJX-104-TEX-N-3D",[171,8841],{"id":8842,"d":8843},"MJX-104-TEX-SO-2211","M61 748Q64 750 489 750H913L954 640Q965 609 976 579T993 533T999 516H979L959 517Q936 579 886 621T777 682Q724 700 655 705T436 710H319Q183 710 183 709Q186 706 348 484T511 259Q517 250 513 244L490 216Q466 188 420 134T330 27L149 -187Q149 -188 362 -188Q388 -188 436 -188T506 -189Q679 -189 778 -162T936 -43Q946 -27 959 6H999L913 -249L489 -250Q65 -250 62 -248Q56 -246 56 -239Q56 -234 118 -161Q186 -81 245 -11L428 206Q428 207 242 462L57 717L56 728Q56 744 61 748Z",[171,8845],{"id":8846,"d":8847},"MJX-104-TEX-N-5B","M118 -250V750H255V710H158V-210H255V-250H118Z",[171,8849],{"id":8850,"d":674},"MJX-104-TEX-I-1D466",[171,8852],{"id":8853,"d":6397},"MJX-104-TEX-N-6C",[171,8855],{"id":8856,"d":8308},"MJX-104-TEX-N-6F",[171,8858],{"id":8859,"d":6409},"MJX-104-TEX-N-67",[171,8861],{"id":8862,"d":5496},"MJX-104-TEX-N-2061",[171,8864],{"id":8865,"d":858},"MJX-104-TEX-I-210E",[171,8867],{"id":8868,"d":178},"MJX-104-TEX-I-1D465",[171,8870],{"id":8871,"d":1051},"MJX-104-TEX-N-2B",[171,8873],{"id":8874,"d":1055},"MJX-104-TEX-N-31",[171,8876],{"id":8877,"d":182},"MJX-104-TEX-N-2212",[171,8879],{"id":8880,"d":8881},"MJX-104-TEX-N-5D","M22 710V750H159V-250H22V-210H119V710H22Z",[208,8883,8884],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,8885,8886,8891,8896,8901,8906,8911,8916],{"dataMmlNode":215},[208,8887,8888],{"dataMmlNode":226},[220,8889],{"dataC":8341,"xLinkHref":8890},"#MJX-104-TEX-I-2113",[208,8892,8893],{"dataMmlNode":218,"transform":8345},[220,8894],{"dataC":222,"xLinkHref":8895},"#MJX-104-TEX-N-28",[208,8897,8898],{"dataMmlNode":226,"transform":8351},[220,8899],{"dataC":1099,"xLinkHref":8900},"#MJX-104-TEX-I-1D703",[208,8902,8903],{"dataMmlNode":218,"transform":8357},[220,8904],{"dataC":265,"xLinkHref":8905},"#MJX-104-TEX-N-29",[208,8907,8908],{"dataMmlNode":218,"transform":8363},[220,8909],{"dataC":1089,"xLinkHref":8910},"#MJX-104-TEX-N-3D",[208,8912,8913],{"dataMmlNode":218,"transform":8369},[220,8914],{"dataC":1353,"xLinkHref":8915},"#MJX-104-TEX-SO-2211",[208,8917,8919,8924,8929,8941,8947,8951,8957,8962,8968,8973,8978,8984,8989,8995,9001,9006,9011,9020,9025,9029,9034,9039,9044,9049,9054,9059,9064],{"dataMmlNode":1821,"transform":8918},"translate(4220.2,0)",[208,8920,8921],{"dataMmlNode":218},[220,8922],{"dataC":4334,"xLinkHref":8923},"#MJX-104-TEX-N-5B",[208,8925,8926],{"dataMmlNode":226,"transform":1788},[220,8927],{"dataC":694,"xLinkHref":8928},"#MJX-104-TEX-I-1D466",[208,8930,8932,8935,8938],{"dataMmlNode":226,"transform":8931},"translate(934.7,0)",[220,8933],{"dataC":6437,"xLinkHref":8934},"#MJX-104-TEX-N-6C",[220,8936],{"dataC":8417,"xLinkHref":8937,"transform":1788},"#MJX-104-TEX-N-6F",[220,8939],{"dataC":6449,"xLinkHref":8940,"transform":6434},"#MJX-104-TEX-N-67",[208,8942,8944],{"dataMmlNode":218,"transform":8943},"translate(2212.7,0)",[220,8945],{"dataC":5574,"xLinkHref":8946},"#MJX-104-TEX-N-2061",[208,8948,8949],{"dataMmlNode":218,"transform":8943},[220,8950],{"dataC":222,"xLinkHref":8895},[208,8952,8954],{"dataMmlNode":226,"transform":8953},"translate(2601.7,0)",[220,8955],{"dataC":876,"xLinkHref":8956},"#MJX-104-TEX-I-210E",[208,8958,8960],{"dataMmlNode":218,"transform":8959},"translate(3177.7,0)",[220,8961],{"dataC":222,"xLinkHref":8895},[208,8963,8965],{"dataMmlNode":226,"transform":8964},"translate(3566.7,0)",[220,8966],{"dataC":230,"xLinkHref":8967},"#MJX-104-TEX-I-1D465",[208,8969,8971],{"dataMmlNode":218,"transform":8970},"translate(4138.7,0)",[220,8972],{"dataC":265,"xLinkHref":8905},[208,8974,8976],{"dataMmlNode":218,"transform":8975},"translate(4527.7,0)",[220,8977],{"dataC":265,"xLinkHref":8905},[208,8979,8981],{"dataMmlNode":218,"transform":8980},"translate(5138.9,0)",[220,8982],{"dataC":1114,"xLinkHref":8983},"#MJX-104-TEX-N-2B",[208,8985,8987],{"dataMmlNode":218,"transform":8986},"translate(6139.1,0)",[220,8988],{"dataC":222,"xLinkHref":8895},[208,8990,8992],{"dataMmlNode":1103,"transform":8991},"translate(6528.1,0)",[220,8993],{"dataC":1127,"xLinkHref":8994},"#MJX-104-TEX-N-31",[208,8996,8998],{"dataMmlNode":218,"transform":8997},"translate(7250.3,0)",[220,8999],{"dataC":237,"xLinkHref":9000},"#MJX-104-TEX-N-2212",[208,9002,9004],{"dataMmlNode":226,"transform":9003},"translate(8250.6,0)",[220,9005],{"dataC":694,"xLinkHref":8928},[208,9007,9009],{"dataMmlNode":218,"transform":9008},"translate(8740.6,0)",[220,9010],{"dataC":265,"xLinkHref":8905},[208,9012,9014,9016,9018],{"dataMmlNode":226,"transform":9013},"translate(9296.2,0)",[220,9015],{"dataC":6437,"xLinkHref":8934},[220,9017],{"dataC":8417,"xLinkHref":8937,"transform":1788},[220,9019],{"dataC":6449,"xLinkHref":8940,"transform":6434},[208,9021,9023],{"dataMmlNode":218,"transform":9022},"translate(10574.2,0)",[220,9024],{"dataC":5574,"xLinkHref":8946},[208,9026,9027],{"dataMmlNode":218,"transform":9022},[220,9028],{"dataC":222,"xLinkHref":8895},[208,9030,9032],{"dataMmlNode":1103,"transform":9031},"translate(10963.2,0)",[220,9033],{"dataC":1127,"xLinkHref":8994},[208,9035,9037],{"dataMmlNode":218,"transform":9036},"translate(11685.4,0)",[220,9038],{"dataC":237,"xLinkHref":9000},[208,9040,9042],{"dataMmlNode":226,"transform":9041},"translate(12685.7,0)",[220,9043],{"dataC":876,"xLinkHref":8956},[208,9045,9047],{"dataMmlNode":218,"transform":9046},"translate(13261.7,0)",[220,9048],{"dataC":222,"xLinkHref":8895},[208,9050,9052],{"dataMmlNode":226,"transform":9051},"translate(13650.7,0)",[220,9053],{"dataC":230,"xLinkHref":8967},[208,9055,9057],{"dataMmlNode":218,"transform":9056},"translate(14222.7,0)",[220,9058],{"dataC":265,"xLinkHref":8905},[208,9060,9062],{"dataMmlNode":218,"transform":9061},"translate(14611.7,0)",[220,9063],{"dataC":265,"xLinkHref":8905},[208,9065,9067],{"dataMmlNode":218,"transform":9066},"translate(15000.7,0)",[220,9068],{"dataC":4445,"xLinkHref":9069},"#MJX-104-TEX-N-5D",[16,9071,9072],{},"This equation looks terrifying, but it's actually an elegant logical \"IF\" statement:",[520,9074,9075,9342],{},[523,9076,9077,1498,9080,9117,9118,9170,9171,9190,9191,9277,9278,9321,9322,9341],{},[20,9078,9079],{},"If the actual answer is",[20,9081,9082,4049],{},[151,9083,9085],{"className":9084,"jax":155},[154],[157,9086,9087,9098],{"style":665,"xmlns":160,"width":7188,"height":7189,"role":163,"focusable":164,"viewBox":7190,"xmlnsXLink":166},[168,9088,9089,9092,9095],{},[171,9090],{"id":9091,"d":674},"MJX-105-TEX-I-1D466",[171,9093],{"id":9094,"d":1039},"MJX-105-TEX-N-3D",[171,9096],{"id":9097,"d":1055},"MJX-105-TEX-N-31",[208,9099,9100],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,9101,9102,9107,9112],{"dataMmlNode":215},[208,9103,9104],{"dataMmlNode":226},[220,9105],{"dataC":694,"xLinkHref":9106},"#MJX-105-TEX-I-1D466",[208,9108,9109],{"dataMmlNode":218,"transform":7213},[220,9110],{"dataC":1089,"xLinkHref":9111},"#MJX-105-TEX-N-3D",[208,9113,9114],{"dataMmlNode":1103,"transform":7219},[220,9115],{"dataC":1127,"xLinkHref":9116},"#MJX-105-TEX-N-31"," The second half of the equation ",[151,9119,9121],{"className":9120,"jax":155},[154],[157,9122,9125,9139],{"style":159,"xmlns":160,"width":9123,"height":162,"role":163,"focusable":164,"viewBox":9124,"xmlnsXLink":166},"6.788ex","0 -750 3000.4 1000",[168,9126,9127,9130,9133,9136],{},[171,9128],{"id":9129,"d":174},"MJX-106-TEX-N-28",[171,9131],{"id":9132,"d":1055},"MJX-106-TEX-N-31",[171,9134],{"id":9135,"d":182},"MJX-106-TEX-N-2212",[171,9137],{"id":9138,"d":198},"MJX-106-TEX-N-29",[208,9140,9141],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,9142,9143,9148,9153,9159,9164],{"dataMmlNode":215},[208,9144,9145],{"dataMmlNode":218},[220,9146],{"dataC":222,"xLinkHref":9147},"#MJX-106-TEX-N-28",[208,9149,9150],{"dataMmlNode":1103,"transform":227},[220,9151],{"dataC":1127,"xLinkHref":9152},"#MJX-106-TEX-N-31",[208,9154,9156],{"dataMmlNode":218,"transform":9155},"translate(1111.2,0)",[220,9157],{"dataC":237,"xLinkHref":9158},"#MJX-106-TEX-N-2212",[208,9160,9162],{"dataMmlNode":1103,"transform":9161},"translate(2111.4,0)",[220,9163],{"dataC":1127,"xLinkHref":9152},[208,9165,9167],{"dataMmlNode":218,"transform":9166},"translate(2611.4,0)",[220,9168],{"dataC":265,"xLinkHref":9169},"#MJX-106-TEX-N-29"," becomes ",[151,9172,9174],{"className":9173,"jax":155},[154],[157,9175,9176,9181],{"style":3975,"xmlns":160,"width":3337,"height":3976,"role":163,"focusable":164,"viewBox":3977,"xmlnsXLink":166},[168,9177,9178],{},[171,9179],{"id":9180,"d":1047},"MJX-107-TEX-N-30",[208,9182,9183],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,9184,9185],{"dataMmlNode":215},[208,9186,9187],{"dataMmlNode":1103},[220,9188],{"dataC":1107,"xLinkHref":9189},"#MJX-107-TEX-N-30"," and disappears. We are just left with ",[151,9192,9194],{"className":9193,"jax":155},[154],[157,9195,9198,9224],{"style":159,"xmlns":160,"width":9196,"height":162,"role":163,"focusable":164,"viewBox":9197,"xmlnsXLink":166},"9.009ex","0 -750 3982 1000",[168,9199,9200,9203,9206,9209,9212,9215,9218,9221],{},[171,9201],{"id":9202,"d":6397},"MJX-108-TEX-N-6C",[171,9204],{"id":9205,"d":8308},"MJX-108-TEX-N-6F",[171,9207],{"id":9208,"d":6409},"MJX-108-TEX-N-67",[171,9210],{"id":9211,"d":5496},"MJX-108-TEX-N-2061",[171,9213],{"id":9214,"d":174},"MJX-108-TEX-N-28",[171,9216],{"id":9217,"d":858},"MJX-108-TEX-I-210E",[171,9219],{"id":9220,"d":178},"MJX-108-TEX-I-1D465",[171,9222],{"id":9223,"d":198},"MJX-108-TEX-N-29",[208,9225,9226],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,9227,9228,9239,9244,9249,9255,9260,9266,9272],{"dataMmlNode":215},[208,9229,9230,9233,9236],{"dataMmlNode":226},[220,9231],{"dataC":6437,"xLinkHref":9232},"#MJX-108-TEX-N-6C",[220,9234],{"dataC":8417,"xLinkHref":9235,"transform":1788},"#MJX-108-TEX-N-6F",[220,9237],{"dataC":6449,"xLinkHref":9238,"transform":6434},"#MJX-108-TEX-N-67",[208,9240,9241],{"dataMmlNode":218,"transform":3630},[220,9242],{"dataC":5574,"xLinkHref":9243},"#MJX-108-TEX-N-2061",[208,9245,9246],{"dataMmlNode":218,"transform":3630},[220,9247],{"dataC":222,"xLinkHref":9248},"#MJX-108-TEX-N-28",[208,9250,9252],{"dataMmlNode":226,"transform":9251},"translate(1667,0)",[220,9253],{"dataC":876,"xLinkHref":9254},"#MJX-108-TEX-I-210E",[208,9256,9258],{"dataMmlNode":218,"transform":9257},"translate(2243,0)",[220,9259],{"dataC":222,"xLinkHref":9248},[208,9261,9263],{"dataMmlNode":226,"transform":9262},"translate(2632,0)",[220,9264],{"dataC":230,"xLinkHref":9265},"#MJX-108-TEX-I-1D465",[208,9267,9269],{"dataMmlNode":218,"transform":9268},"translate(3204,0)",[220,9270],{"dataC":265,"xLinkHref":9271},"#MJX-108-TEX-N-29",[208,9273,9275],{"dataMmlNode":218,"transform":9274},"translate(3593,0)",[220,9276],{"dataC":265,"xLinkHref":9271},". We want our prediction ",[151,9279,9281],{"className":9280,"jax":155},[154],[157,9282,9283,9297],{"style":159,"xmlns":160,"width":851,"height":162,"role":163,"focusable":164,"viewBox":852,"xmlnsXLink":166},[168,9284,9285,9288,9291,9294],{},[171,9286],{"id":9287,"d":858},"MJX-109-TEX-I-210E",[171,9289],{"id":9290,"d":174},"MJX-109-TEX-N-28",[171,9292],{"id":9293,"d":178},"MJX-109-TEX-I-1D465",[171,9295],{"id":9296,"d":198},"MJX-109-TEX-N-29",[208,9298,9299],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,9300,9301,9306,9311,9316],{"dataMmlNode":215},[208,9302,9303],{"dataMmlNode":226},[220,9304],{"dataC":876,"xLinkHref":9305},"#MJX-109-TEX-I-210E",[208,9307,9308],{"dataMmlNode":218,"transform":880},[220,9309],{"dataC":222,"xLinkHref":9310},"#MJX-109-TEX-N-28",[208,9312,9313],{"dataMmlNode":226,"transform":886},[220,9314],{"dataC":230,"xLinkHref":9315},"#MJX-109-TEX-I-1D465",[208,9317,9318],{"dataMmlNode":218,"transform":892},[220,9319],{"dataC":265,"xLinkHref":9320},"#MJX-109-TEX-N-29"," to be as close to ",[151,9323,9325],{"className":9324,"jax":155},[154],[157,9326,9327,9332],{"style":3336,"xmlns":160,"width":3337,"height":3338,"role":163,"focusable":164,"viewBox":3339,"xmlnsXLink":166},[168,9328,9329],{},[171,9330],{"id":9331,"d":1055},"MJX-110-TEX-N-31",[208,9333,9334],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,9335,9336],{"dataMmlNode":215},[208,9337,9338],{"dataMmlNode":1103},[220,9339],{"dataC":1127,"xLinkHref":9340},"#MJX-110-TEX-N-31"," as possible.",[523,9343,9344,1498,9346,9383,9384,9170,9421,9190,9440,9277,9543,9321,9586,9341],{},[20,9345,9079],{},[20,9347,9348,4049],{},[151,9349,9351],{"className":9350,"jax":155},[154],[157,9352,9353,9364],{"style":665,"xmlns":160,"width":7188,"height":7189,"role":163,"focusable":164,"viewBox":7190,"xmlnsXLink":166},[168,9354,9355,9358,9361],{},[171,9356],{"id":9357,"d":674},"MJX-111-TEX-I-1D466",[171,9359],{"id":9360,"d":1039},"MJX-111-TEX-N-3D",[171,9362],{"id":9363,"d":1047},"MJX-111-TEX-N-30",[208,9365,9366],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,9367,9368,9373,9378],{"dataMmlNode":215},[208,9369,9370],{"dataMmlNode":226},[220,9371],{"dataC":694,"xLinkHref":9372},"#MJX-111-TEX-I-1D466",[208,9374,9375],{"dataMmlNode":218,"transform":7213},[220,9376],{"dataC":1089,"xLinkHref":9377},"#MJX-111-TEX-N-3D",[208,9379,9380],{"dataMmlNode":1103,"transform":7219},[220,9381],{"dataC":1107,"xLinkHref":9382},"#MJX-111-TEX-N-30"," The first half of the equation ",[151,9385,9387],{"className":9386,"jax":155},[154],[157,9388,9390,9401],{"style":159,"xmlns":160,"width":7301,"height":162,"role":163,"focusable":164,"viewBox":9389,"xmlnsXLink":166},"0 -750 1278 1000",[168,9391,9392,9395,9398],{},[171,9393],{"id":9394,"d":174},"MJX-112-TEX-N-28",[171,9396],{"id":9397,"d":1047},"MJX-112-TEX-N-30",[171,9399],{"id":9400,"d":198},"MJX-112-TEX-N-29",[208,9402,9403],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,9404,9405,9410,9415],{"dataMmlNode":215},[208,9406,9407],{"dataMmlNode":218},[220,9408],{"dataC":222,"xLinkHref":9409},"#MJX-112-TEX-N-28",[208,9411,9412],{"dataMmlNode":1103,"transform":227},[220,9413],{"dataC":1107,"xLinkHref":9414},"#MJX-112-TEX-N-30",[208,9416,9418],{"dataMmlNode":218,"transform":9417},"translate(889,0)",[220,9419],{"dataC":265,"xLinkHref":9420},"#MJX-112-TEX-N-29",[151,9422,9424],{"className":9423,"jax":155},[154],[157,9425,9426,9431],{"style":3975,"xmlns":160,"width":3337,"height":3976,"role":163,"focusable":164,"viewBox":3977,"xmlnsXLink":166},[168,9427,9428],{},[171,9429],{"id":9430,"d":1047},"MJX-113-TEX-N-30",[208,9432,9433],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,9434,9435],{"dataMmlNode":215},[208,9436,9437],{"dataMmlNode":1103},[220,9438],{"dataC":1107,"xLinkHref":9439},"#MJX-113-TEX-N-30",[151,9441,9443],{"className":9442,"jax":155},[154],[157,9444,9447,9479],{"style":159,"xmlns":160,"width":9445,"height":162,"role":163,"focusable":164,"viewBox":9446,"xmlnsXLink":166},"12.906ex","0 -750 5704.4 1000",[168,9448,9449,9452,9455,9458,9461,9464,9467,9470,9473,9476],{},[171,9450],{"id":9451,"d":6397},"MJX-114-TEX-N-6C",[171,9453],{"id":9454,"d":8308},"MJX-114-TEX-N-6F",[171,9456],{"id":9457,"d":6409},"MJX-114-TEX-N-67",[171,9459],{"id":9460,"d":5496},"MJX-114-TEX-N-2061",[171,9462],{"id":9463,"d":174},"MJX-114-TEX-N-28",[171,9465],{"id":9466,"d":1055},"MJX-114-TEX-N-31",[171,9468],{"id":9469,"d":182},"MJX-114-TEX-N-2212",[171,9471],{"id":9472,"d":858},"MJX-114-TEX-I-210E",[171,9474],{"id":9475,"d":178},"MJX-114-TEX-I-1D465",[171,9477],{"id":9478,"d":198},"MJX-114-TEX-N-29",[208,9480,9481],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,9482,9483,9494,9499,9504,9509,9515,9521,9526,9532,9538],{"dataMmlNode":215},[208,9484,9485,9488,9491],{"dataMmlNode":226},[220,9486],{"dataC":6437,"xLinkHref":9487},"#MJX-114-TEX-N-6C",[220,9489],{"dataC":8417,"xLinkHref":9490,"transform":1788},"#MJX-114-TEX-N-6F",[220,9492],{"dataC":6449,"xLinkHref":9493,"transform":6434},"#MJX-114-TEX-N-67",[208,9495,9496],{"dataMmlNode":218,"transform":3630},[220,9497],{"dataC":5574,"xLinkHref":9498},"#MJX-114-TEX-N-2061",[208,9500,9501],{"dataMmlNode":218,"transform":3630},[220,9502],{"dataC":222,"xLinkHref":9503},"#MJX-114-TEX-N-28",[208,9505,9506],{"dataMmlNode":1103,"transform":9251},[220,9507],{"dataC":1127,"xLinkHref":9508},"#MJX-114-TEX-N-31",[208,9510,9512],{"dataMmlNode":218,"transform":9511},"translate(2389.2,0)",[220,9513],{"dataC":237,"xLinkHref":9514},"#MJX-114-TEX-N-2212",[208,9516,9518],{"dataMmlNode":226,"transform":9517},"translate(3389.4,0)",[220,9519],{"dataC":876,"xLinkHref":9520},"#MJX-114-TEX-I-210E",[208,9522,9524],{"dataMmlNode":218,"transform":9523},"translate(3965.4,0)",[220,9525],{"dataC":222,"xLinkHref":9503},[208,9527,9529],{"dataMmlNode":226,"transform":9528},"translate(4354.4,0)",[220,9530],{"dataC":230,"xLinkHref":9531},"#MJX-114-TEX-I-1D465",[208,9533,9535],{"dataMmlNode":218,"transform":9534},"translate(4926.4,0)",[220,9536],{"dataC":265,"xLinkHref":9537},"#MJX-114-TEX-N-29",[208,9539,9541],{"dataMmlNode":218,"transform":9540},"translate(5315.4,0)",[220,9542],{"dataC":265,"xLinkHref":9537},[151,9544,9546],{"className":9545,"jax":155},[154],[157,9547,9548,9562],{"style":159,"xmlns":160,"width":851,"height":162,"role":163,"focusable":164,"viewBox":852,"xmlnsXLink":166},[168,9549,9550,9553,9556,9559],{},[171,9551],{"id":9552,"d":858},"MJX-115-TEX-I-210E",[171,9554],{"id":9555,"d":174},"MJX-115-TEX-N-28",[171,9557],{"id":9558,"d":178},"MJX-115-TEX-I-1D465",[171,9560],{"id":9561,"d":198},"MJX-115-TEX-N-29",[208,9563,9564],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,9565,9566,9571,9576,9581],{"dataMmlNode":215},[208,9567,9568],{"dataMmlNode":226},[220,9569],{"dataC":876,"xLinkHref":9570},"#MJX-115-TEX-I-210E",[208,9572,9573],{"dataMmlNode":218,"transform":880},[220,9574],{"dataC":222,"xLinkHref":9575},"#MJX-115-TEX-N-28",[208,9577,9578],{"dataMmlNode":226,"transform":886},[220,9579],{"dataC":230,"xLinkHref":9580},"#MJX-115-TEX-I-1D465",[208,9582,9583],{"dataMmlNode":218,"transform":892},[220,9584],{"dataC":265,"xLinkHref":9585},"#MJX-115-TEX-N-29",[151,9587,9589],{"className":9588,"jax":155},[154],[157,9590,9591,9596],{"style":3975,"xmlns":160,"width":3337,"height":3976,"role":163,"focusable":164,"viewBox":3977,"xmlnsXLink":166},[168,9592,9593],{},[171,9594],{"id":9595,"d":1047},"MJX-116-TEX-N-30",[208,9597,9598],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,9599,9600],{"dataMmlNode":215},[208,9601,9602],{"dataMmlNode":1103},[220,9603],{"dataC":1107,"xLinkHref":9604},"#MJX-116-TEX-N-30",[16,9606,9607,9608,9611],{},"It essentially ",[20,9609,9610],{},"heavily punishes the model if it is very confident but wrong."," (e.g., If the model predicts a 99% probability of a tumor being Malignant, but it's actually Benign, the cost explodes to infinity).",[28,9613,9615],{"id":9614},"the-beautiful-symmetry","The \"Beautiful Symmetry\"",[16,9617,9618],{},"Here is the craziest part of all of this.",[16,9620,9621],{},"If you take that terrifying Log-Likelihood equation, and you do the calculus chain rule to find the Gradient Descent update rule (just like we did for linear regression)... a bunch of math cancels out perfectly.",[16,9623,9624,9625],{},"You are left with this:\n",[151,9626,9628],{"className":9627,"jax":155},[154],[157,9629,9630,9671],{"style":1965,"xmlns":160,"width":1966,"height":1967,"role":163,"focusable":164,"viewBox":1968,"xmlnsXLink":166},[168,9631,9632,9635,9638,9641,9644,9647,9650,9653,9656,9659,9662,9665,9668],{},[171,9633],{"id":9634,"d":1043},"MJX-117-TEX-I-1D703",[171,9636],{"id":9637,"d":1732},"MJX-117-TEX-I-1D457",[171,9639],{"id":9640,"d":1736},"MJX-117-TEX-N-3A",[171,9642],{"id":9643,"d":1039},"MJX-117-TEX-N-3D",[171,9645],{"id":9646,"d":1051},"MJX-117-TEX-N-2B",[171,9648],{"id":9649,"d":1746},"MJX-117-TEX-I-1D6FC",[171,9651],{"id":9652,"d":174},"MJX-117-TEX-N-28",[171,9654],{"id":9655,"d":674},"MJX-117-TEX-I-1D466",[171,9657],{"id":9658,"d":190},"MJX-117-TEX-I-1D456",[171,9660],{"id":9661,"d":198},"MJX-117-TEX-N-29",[171,9663],{"id":9664,"d":182},"MJX-117-TEX-N-2212",[171,9666],{"id":9667,"d":858},"MJX-117-TEX-I-210E",[171,9669],{"id":9670,"d":178},"MJX-117-TEX-I-1D465",[208,9672,9673],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,9674,9675,9687,9699,9709,9714,9719,9724,9747,9752,9757,9761,9782,9786,9790],{"dataMmlNode":215},[208,9676,9677,9682],{"dataMmlNode":1093},[208,9678,9679],{"dataMmlNode":226},[220,9680],{"dataC":1099,"xLinkHref":9681},"#MJX-117-TEX-I-1D703",[208,9683,9684],{"dataMmlNode":226,"transform":1104},[220,9685],{"dataC":1775,"xLinkHref":9686},"#MJX-117-TEX-I-1D457",[208,9688,9689,9694],{"dataMmlNode":218,"transform":1779},[208,9690,9691],{"dataMmlNode":980},[220,9692],{"dataC":1784,"xLinkHref":9693},"#MJX-117-TEX-N-3A",[208,9695,9696],{"dataMmlNode":980,"transform":1788},[220,9697],{"dataC":1089,"xLinkHref":9698},"#MJX-117-TEX-N-3D",[208,9700,9701,9705],{"dataMmlNode":1093,"transform":1794},[208,9702,9703],{"dataMmlNode":226},[220,9704],{"dataC":1099,"xLinkHref":9681},[208,9706,9707],{"dataMmlNode":226,"transform":1104},[220,9708],{"dataC":1775,"xLinkHref":9686},[208,9710,9711],{"dataMmlNode":218,"transform":1805},[220,9712],{"dataC":1114,"xLinkHref":9713},"#MJX-117-TEX-N-2B",[208,9715,9716],{"dataMmlNode":226,"transform":1811},[220,9717],{"dataC":1814,"xLinkHref":9718},"#MJX-117-TEX-I-1D6FC",[208,9720,9721],{"dataMmlNode":218,"transform":1818},[220,9722],{"dataC":222,"xLinkHref":9723},"#MJX-117-TEX-N-28",[208,9725,9726,9731],{"dataMmlNode":630,"transform":2065},[208,9727,9728],{"dataMmlNode":226},[220,9729],{"dataC":694,"xLinkHref":9730},"#MJX-117-TEX-I-1D466",[208,9732,9733,9737,9742],{"dataMmlNode":269,"transform":698,"dataMjxTexclass":270},[208,9734,9735],{"dataMmlNode":218},[220,9736],{"dataC":222,"xLinkHref":9723},[208,9738,9739],{"dataMmlNode":226,"transform":227},[220,9740],{"dataC":251,"xLinkHref":9741},"#MJX-117-TEX-I-1D456",[208,9743,9744],{"dataMmlNode":218,"transform":651},[220,9745],{"dataC":265,"xLinkHref":9746},"#MJX-117-TEX-N-29",[208,9748,9749],{"dataMmlNode":218,"transform":2089},[220,9750],{"dataC":237,"xLinkHref":9751},"#MJX-117-TEX-N-2212",[208,9753,9754],{"dataMmlNode":226,"transform":2095},[220,9755],{"dataC":876,"xLinkHref":9756},"#MJX-117-TEX-I-210E",[208,9758,9759],{"dataMmlNode":218,"transform":2101},[220,9760],{"dataC":222,"xLinkHref":9723},[208,9762,9763,9768],{"dataMmlNode":630,"transform":2106},[208,9764,9765],{"dataMmlNode":226},[220,9766],{"dataC":230,"xLinkHref":9767},"#MJX-117-TEX-I-1D465",[208,9769,9770,9774,9778],{"dataMmlNode":269,"transform":638,"dataMjxTexclass":270},[208,9771,9772],{"dataMmlNode":218},[220,9773],{"dataC":222,"xLinkHref":9723},[208,9775,9776],{"dataMmlNode":226,"transform":227},[220,9777],{"dataC":251,"xLinkHref":9741},[208,9779,9780],{"dataMmlNode":218,"transform":651},[220,9781],{"dataC":265,"xLinkHref":9746},[208,9783,9784],{"dataMmlNode":218,"transform":2128},[220,9785],{"dataC":265,"xLinkHref":9746},[208,9787,9788],{"dataMmlNode":218,"transform":2133},[220,9789],{"dataC":265,"xLinkHref":9746},[208,9791,9792,9796,9810],{"dataMmlNode":2138,"transform":2139},[208,9793,9794],{"dataMmlNode":226},[220,9795],{"dataC":230,"xLinkHref":9767},[208,9797,9798,9802,9806],{"dataMmlNode":269,"transform":2146,"dataMjxTexclass":270},[208,9799,9800],{"dataMmlNode":218},[220,9801],{"dataC":222,"xLinkHref":9723},[208,9803,9804],{"dataMmlNode":226,"transform":227},[220,9805],{"dataC":251,"xLinkHref":9741},[208,9807,9808],{"dataMmlNode":218,"transform":651},[220,9809],{"dataC":265,"xLinkHref":9746},[208,9811,9812],{"dataMmlNode":226,"transform":2161},[220,9813],{"dataC":1775,"xLinkHref":9686},[16,9815,9816],{},[20,9817,9818],{},"It is the EXACT same update rule as standard Linear Regression.",[16,9820,9821,9822,9825,9826,9869,9870,1542],{},"The computer code you write to update the weights for Logistic Regression is identical to the code for Linear Regression. The ",[87,9823,9824],{},"only"," difference is that in Logistic Regression, ",[151,9827,9829],{"className":9828,"jax":155},[154],[157,9830,9831,9845],{"style":159,"xmlns":160,"width":851,"height":162,"role":163,"focusable":164,"viewBox":852,"xmlnsXLink":166},[168,9832,9833,9836,9839,9842],{},[171,9834],{"id":9835,"d":858},"MJX-118-TEX-I-210E",[171,9837],{"id":9838,"d":174},"MJX-118-TEX-N-28",[171,9840],{"id":9841,"d":178},"MJX-118-TEX-I-1D465",[171,9843],{"id":9844,"d":198},"MJX-118-TEX-N-29",[208,9846,9847],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,9848,9849,9854,9859,9864],{"dataMmlNode":215},[208,9850,9851],{"dataMmlNode":226},[220,9852],{"dataC":876,"xLinkHref":9853},"#MJX-118-TEX-I-210E",[208,9855,9856],{"dataMmlNode":218,"transform":880},[220,9857],{"dataC":222,"xLinkHref":9858},"#MJX-118-TEX-N-28",[208,9860,9861],{"dataMmlNode":226,"transform":886},[220,9862],{"dataC":230,"xLinkHref":9863},"#MJX-118-TEX-I-1D465",[208,9865,9866],{"dataMmlNode":218,"transform":892},[220,9867],{"dataC":265,"xLinkHref":9868},"#MJX-118-TEX-N-29"," calculates the Sigmoid formula instead of just ",[151,9871,9873],{"className":9872,"jax":155},[154],[157,9874,9878,9899],{"style":665,"xmlns":160,"width":9875,"height":9876,"role":163,"focusable":164,"viewBox":9877,"xmlnsXLink":166},"11.152ex","1.783ex","0 -583 4929 788",[168,9879,9880,9883,9886,9889,9892,9895],{},[171,9881],{"id":9882,"d":674},"MJX-119-TEX-I-1D466",[171,9884],{"id":9885,"d":1039},"MJX-119-TEX-N-3D",[171,9887],{"id":9888,"d":186},"MJX-119-TEX-I-1D45A",[171,9890],{"id":9891,"d":178},"MJX-119-TEX-I-1D465",[171,9893],{"id":9894,"d":1051},"MJX-119-TEX-N-2B",[171,9896],{"id":9897,"d":9898},"MJX-119-TEX-I-1D450","M34 159Q34 268 120 355T306 442Q362 442 394 418T427 355Q427 326 408 306T360 285Q341 285 330 295T319 325T330 359T352 380T366 386H367Q367 388 361 392T340 400T306 404Q276 404 249 390Q228 381 206 359Q162 315 142 235T121 119Q121 73 147 50Q169 26 205 26H209Q321 26 394 111Q403 121 406 121Q410 121 419 112T429 98T420 83T391 55T346 25T282 0T202 -11Q127 -11 81 37T34 159Z",[208,9900,9901],{"stroke":210,"fill":210,"stroke-width":211,"transform":212},[208,9902,9903,9908,9913,9918,9924,9930],{"dataMmlNode":215},[208,9904,9905],{"dataMmlNode":226},[220,9906],{"dataC":694,"xLinkHref":9907},"#MJX-119-TEX-I-1D466",[208,9909,9910],{"dataMmlNode":218,"transform":7213},[220,9911],{"dataC":1089,"xLinkHref":9912},"#MJX-119-TEX-N-3D",[208,9914,9915],{"dataMmlNode":226,"transform":7219},[220,9916],{"dataC":244,"xLinkHref":9917},"#MJX-119-TEX-I-1D45A",[208,9919,9921],{"dataMmlNode":226,"transform":9920},"translate(2701.6,0)",[220,9922],{"dataC":230,"xLinkHref":9923},"#MJX-119-TEX-I-1D465",[208,9925,9927],{"dataMmlNode":218,"transform":9926},"translate(3495.8,0)",[220,9928],{"dataC":1114,"xLinkHref":9929},"#MJX-119-TEX-N-2B",[208,9931,9933],{"dataMmlNode":226,"transform":9932},"translate(4496,0)",[220,9934],{"dataC":9935,"xLinkHref":9936},"1D450","#MJX-119-TEX-I-1D450",[11,9938,9939],{"id":75},"Summary",[9941,9942,9943,9956],"table",{},[9944,9945,9946],"thead",{},[9947,9948,9949,9953],"tr",{},[9950,9951,9952],"th",{},"Topic",[9950,9954,9955],{},"Key Idea",[9957,9958,9959,9967,9974,9982,9990,9998,10006,10014],"tbody",{},[9947,9960,9961,9964],{},[9962,9963,14],"td",{},[9962,9965,9966],{},"Clean and transform raw data before training",[9947,9968,9969,9971],{},[9962,9970,947],{},[9962,9972,9973],{},"Predict continuous values with a linear model",[9947,9975,9976,9979],{},[9962,9977,9978],{},"Cost Function (J)",[9962,9980,9981],{},"Measures how wrong our predictions are",[9947,9983,9984,9987],{},[9962,9985,9986],{},"Gradient Descent",[9962,9988,9989],{},"Iteratively adjust parameters to minimize cost",[9947,9991,9992,9995],{},[9962,9993,9994],{},"Batch vs. Stochastic GD",[9962,9996,9997],{},"Update after all examples vs. after each example",[9947,9999,10000,10003],{},[9962,10001,10002],{},"Underfitting \u002F Overfitting",[9962,10004,10005],{},"Too simple vs. too complex models",[9947,10007,10008,10011],{},[9962,10009,10010],{},"Locally Weighted LR",[9962,10012,10013],{},"Give nearby points more influence",[9947,10015,10016,10018],{},[9962,10017,6922],{},[9962,10019,10020],{},"Classify discrete outcomes using the sigmoid function",[10022,10023,10024],"style",{},"\nmjx-container[jax=\"SVG\"] {\n  direction: ltr;\n}\n\nmjx-container[jax=\"SVG\"] > svg {\n  overflow: visible;\n  min-height: 1px;\n  min-width: 1px;\n}\n\nmjx-container[jax=\"SVG\"] > svg a {\n  fill: blue;\n  stroke: blue;\n}\n\nmjx-container[jax=\"SVG\"][display=\"true\"] {\n  display: block;\n  text-align: center;\n  margin: 1em 0;\n}\n\nmjx-container[jax=\"SVG\"][display=\"true\"][width=\"full\"] {\n  display: flex;\n}\n\nmjx-container[jax=\"SVG\"][justify=\"left\"] {\n  text-align: left;\n}\n\nmjx-container[jax=\"SVG\"][justify=\"right\"] {\n  text-align: right;\n}\n\ng[data-mml-node=\"merror\"] > g {\n  fill: red;\n  stroke: red;\n}\n\ng[data-mml-node=\"merror\"] > rect[data-background] {\n  fill: yellow;\n  stroke: none;\n}\n\ng[data-mml-node=\"mtable\"] > line[data-line], svg[data-table] > g > line[data-line] {\n  stroke-width: 70px;\n  fill: none;\n}\n\ng[data-mml-node=\"mtable\"] > rect[data-frame], svg[data-table] > g > rect[data-frame] {\n  stroke-width: 70px;\n  fill: none;\n}\n\ng[data-mml-node=\"mtable\"] > .mjx-dashed, svg[data-table] > g > .mjx-dashed {\n  stroke-dasharray: 140;\n}\n\ng[data-mml-node=\"mtable\"] > .mjx-dotted, svg[data-table] > g > .mjx-dotted {\n  stroke-linecap: round;\n  stroke-dasharray: 0,140;\n}\n\ng[data-mml-node=\"mtable\"] > g > svg {\n  overflow: visible;\n}\n\n[jax=\"SVG\"] mjx-tool {\n  display: inline-block;\n  position: relative;\n  width: 0;\n  height: 0;\n}\n\n[jax=\"SVG\"] mjx-tool > mjx-tip {\n  position: absolute;\n  top: 0;\n  left: 0;\n}\n\nmjx-tool > mjx-tip {\n  display: inline-block;\n  padding: .2em;\n  border: 1px solid #888;\n  font-size: 70%;\n  background-color: #F8F8F8;\n  color: black;\n  box-shadow: 2px 2px 5px #AAAAAA;\n}\n\ng[data-mml-node=\"maction\"][data-toggle] {\n  cursor: pointer;\n}\n\nmjx-status {\n  display: block;\n  position: fixed;\n  left: 1em;\n  bottom: 1em;\n  min-width: 25%;\n  padding: .2em .4em;\n  border: 1px solid #888;\n  font-size: 90%;\n  background-color: #F8F8F8;\n  color: black;\n}\n\nforeignObject[data-mjx-xml] {\n  font-family: initial;\n  line-height: normal;\n  overflow: visible;\n}\n\nmjx-container[jax=\"SVG\"] path[data-c], mjx-container[jax=\"SVG\"] use[data-c] {\n  stroke-width: 3;\n}\n",{"title":5496,"searchDepth":10026,"depth":10026,"links":10027},2,[10028,10033,10043,10046],{"id":13,"depth":10026,"text":14,"children":10029},[10030,10032],{"id":30,"depth":10031,"text":31},3,{"id":514,"depth":10031,"text":515},{"id":543,"depth":10026,"text":544,"children":10034},[10035,10036,10037,10038,10039,10040,10041,10042],{"id":550,"depth":10031,"text":551},{"id":590,"depth":10031,"text":591},{"id":946,"depth":10031,"text":947},{"id":1227,"depth":10031,"text":1228},{"id":1569,"depth":10031,"text":1570},{"id":5314,"depth":10031,"text":5315},{"id":5370,"depth":10031,"text":5371},{"id":5444,"depth":10031,"text":5445},{"id":6921,"depth":10026,"text":6922,"children":10044},[10045],{"id":7972,"depth":10031,"text":7973},{"id":75,"depth":10026,"text":9939},"2026-04-27",false,"md",null,{},true,"\u002Fblog\u002FData-Preprocessing-&-Regression-Analysis",{"title":5,"description":5496},{"loc":10053},"blog\u002FData-Preprocessing-&-Regression-Analysis","P6PQEineKgTY9oAO9xFso7XOZj1Ctgl9Wdi0ldlOmPY",104,{"id":10060,"extension":10061,"meta":10062,"series":10063,"stem":10167,"__hash__":10168},"series\u002Fseries.json","json",{},{"微積分教學":10064,"生活紀錄":10067,"Motor Control":10069,"生活隨筆":10083,"Motor learning":10087,"小兒物治":10105,"中風":10120,"平衡":10131,"Network Communication":10141,"CSA":10148,"機器學習":10154,"小腦":10158},[10065,10066],"微積分隨筆-未完成版","2025數學回顧",[10068],"一個漂流到地球的故事",[10070,10071,10072,10073,10074,10075,10076,10077,10078,10079,10080,10081,10082],"控制自己-Be-water-my-friend","控制自己-Be-water-my-friend（二）","控制自己-Be-water-my-friend（三）","控制自己-Be-water-my-friend（四）","控制自己-Be-water-my-friend（五）","進階控制制制制","周圍理論學派（一）反射理論","周圍理論學派（二）階層理論","中樞理論學派（一）CPG","中樞理論學派（二）Motor-Program","模組理論","系統理論","動態模組理論",[10084,10085,10086],"你好，世界。","根本沒人在乎你的部落格","早安-午安-晚安",[10088,10089,10090,10091,10092,10093,10094,10095,10096,10097,10098,10099,10100,10101,10102,10103,10104],"動作學習（一）介紹","動作學習（二）form-of-learning","動作學習（三）Measurement-of-learning","動作學習（四）理論","動作學習（五）理論-2","動作學習（六）理論-3","動作學習（七）練習方式-1","動作學習（八）練習方式-2","動作學習（九）回饋-1","動作學習（十）回饋-2-擴增性(KR)","動作學習（十一）回饋-3-擴增性(KP)","動作學習（十一）回饋-4-(間隔+物理引導)","動作學習（十二）神經可塑性","動作學習（十二）神經可塑性2","動作學習（十三）臨床應用","動作學習（十四）記憶","動作學習（十五）影響表現的因素",[10106,10107,10108,10109,10110,10111,10112,10113,10114,10115,10116,10117,10118,10119],"腦性痲痺-CP","CP補充（一）","CP—Rood-&-Bobath","CP—Rood-&-Bobath（二）","Motor-Learning","Motor-Learning小兒（二）","Gait-analysis小兒（一）","Gait-analysis小兒（二）","小兒發展（一）","小兒發展（二）","小兒發展（三）","小兒發展（四）","小兒發展（五）","GMFCS",[10121,10122,10123,10124,10125,10126,10127,10128,10129,10130],"腦血管病變（CVA）（中風）(一)","CVA（二）","CVA（三）血管症候群-i","CVA（四）血管症候群-(ii)","CVA（四）","CVA（六）","CVA（七）評估-(i)","CVA（八）評估-(ii)","CVA（九）復健—手部-(i)","CVA跑台（一）",[10132,10133,10134,10135,10136,10137,10138,10139,10140],"平衡與前庭失調（一）","Balance（二）前庭覺-(i)","Balance（三）","Balance（四）評估","Balance（五）復健","Balance（六）功能恢復","Balance（七）前庭障礙","Balance（八）檢查","Balance（九）干預",[10142,10143,10144,10145,10146,10147],"Network-Communication,-Chapter-1","Network-Communication,-Chapter-2","Network-Communication,-Chapter-3","Network-Communication-Chapter-4","Network-Communications,-Chapter-5","Network-Communication,-Chapter-6",[10149,10150,10151,10152,10153],"Week-1-—-Introduction-to-Computer-Systems","Computer-Systems-Architecture-Understanding-Performance","A-Top-Level-View-of-Computer-Function-and-Interconnection","The-Memory-Hierarchy-Understanding-Cache-Memory","Internal-Memory-How-Your-Computer-Remembers-Things",[10155,10156,10157],"機器學習導論","資料前處理與迴歸分析","決策樹",[10159,10160,10161,10162,10163,10164,10165,10166],"小腦（一）","小腦（二）","小腦（三）功能","小腦（四）損傷","小腦（五）各功能障礙","小腦（六）評估","小腦（七）評估(ii)","小腦（八）治療","series","sJucyaG7gC0dv353KrH133RVxHX0BiFnZMLor9gTQ3o",[10170,10189,10209,10228,10245,10262,10277,10294,10312,10334],{"id":10171,"title":10172,"avatar":10173,"banner":10050,"bio":10174,"body":10175,"description":5496,"extension":10049,"meta":10179,"name":10172,"navigation":10052,"path":10180,"seo":10181,"sitemap":10182,"social":10183,"stem":10187,"__hash__":10188},"authors\u002Fauthors\u002Fautomata.md","Automata","\u002Fimages\u002Fuploads\u002Fnier-automata-2b.jpg","一隻吐司天喵，漂浮在銀河星辰中",{"type":8,"value":10176,"toc":10177},[],{"title":5496,"searchDepth":10026,"depth":10026,"links":10178},[],{},"\u002Fauthors\u002Fautomata",{"description":5496},{"loc":10180},{"website":10184,"twitter":10185,"github":10186},"https:\u002F\u002Freurl.cc\u002FWOeM29","https:\u002F\u002Freurl.cc\u002FLnvLEy","https:\u002F\u002Fgithub.com\u002FAutomata-0","authors\u002Fautomata","IkVbO2zA7revgYq624iVWpSZQUyMmWa82tw_EbWXViE",{"id":10190,"title":10191,"avatar":10192,"banner":10193,"bio":10194,"body":10195,"description":5496,"extension":10049,"meta":10199,"name":10200,"navigation":10052,"path":10201,"seo":10202,"sitemap":10203,"social":10204,"stem":10207,"__hash__":10208},"authors\u002Fauthors\u002Fchinono.md","Chinono","\u002Fimages\u002Fuploads\u002F103467998_p0 copy.png","\u002Fimages\u002Fbackground_light.jpg","我不是女生！",{"type":8,"value":10196,"toc":10197},[],{"title":5496,"searchDepth":10026,"depth":10026,"links":10198},[],{},"七糯糯","\u002Fauthors\u002Fchinono",{"description":5496},{"loc":10201},{"github":10205,"twitter":5496,"website":10206},"https:\u002F\u002Fgithub.com\u002FChinHongTan","https:\u002F\u002Fchinono.dev","authors\u002Fchinono","jj1J9mFh3InZFL6XtCzGBQ5jPip0EwBDE3mjGvnN6jE",{"id":10210,"title":10211,"avatar":10212,"banner":10213,"bio":10214,"body":10215,"description":5496,"extension":10049,"meta":10219,"name":10220,"navigation":10052,"path":10221,"seo":10222,"sitemap":10223,"social":10224,"stem":10226,"__hash__":10227},"authors\u002Fauthors\u002Fhibiki12141132.md","Hibiki12141132","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F265822020?v=4","\u002Fimages\u002Fuploads\u002F1773978423557-___.jpg","享受著知識強姦大腦的過程 (內文含個人發癲 不要再意)",{"type":8,"value":10216,"toc":10217},[],{"title":5496,"searchDepth":10026,"depth":10026,"links":10218},[],{},"HiBiKi","\u002Fauthors\u002Fhibiki12141132",{"description":5496},{"loc":10221},{"github":10225,"twitter":5496},"https:\u002F\u002Fgithub.com\u002FHiBiKi12141132","authors\u002Fhibiki12141132","dbRnKEcYeCH_faD8R7AUmPPcwgc26s_fR4Q_lu4qtA4",{"id":10229,"title":10230,"avatar":10231,"banner":10050,"bio":10232,"body":10233,"description":5496,"extension":10049,"meta":10237,"name":10230,"navigation":10052,"path":10238,"seo":10239,"sitemap":10240,"social":10241,"stem":10243,"__hash__":10244},"authors\u002Fauthors\u002Fmahiro.md","Mahiro","https:\u002F\u002Ftruth.bahamut.com.tw\u002Fs01\u002F202601\u002F2a29b047d341f840b2ce89f7d65b2ba3.JPG","一個致力於逃離新竹的電機系小雜魚",{"type":8,"value":10234,"toc":10235},[],{"title":5496,"searchDepth":10026,"depth":10026,"links":10236},[],{},"\u002Fauthors\u002Fmahiro",{"description":5496},{"loc":10238},{"github":10242},"https:\u002F\u002Fgithub.com\u002Fwifekurumi","authors\u002Fmahiro","b435tdWu9eXUf06WroCge0I405cqA0FhLlUUhoPk14k",{"id":10246,"title":10247,"avatar":10248,"banner":10050,"bio":10249,"body":10250,"description":5496,"extension":10049,"meta":10254,"name":10247,"navigation":10052,"path":10255,"seo":10256,"sitemap":10257,"social":10258,"stem":10260,"__hash__":10261},"authors\u002Fauthors\u002Fosborrrrn.md","Osborrrrn","\u002Fimages\u002Fuploads\u002Frectangle_large_type_2_c516437ed713e5de1f7d2dca8a20cd81.jpg","別人笑我太瘋癲，我笑他人看不穿。\n不見五陵豪傑墓，無花無酒鋤就田",{"type":8,"value":10251,"toc":10252},[],{"title":5496,"searchDepth":10026,"depth":10026,"links":10253},[],{},"\u002Fauthors\u002Fosborrrrn",{"description":5496},{"loc":10255},{"github":10259},"https:\u002F\u002Fgithub.com\u002FOsborrrrn","authors\u002Fosborrrrn","w6VWZKPUwvXn5i7MKXOpU2Jeqr3BrdTKVCeDOF2jZlU",{"id":10263,"title":10264,"avatar":10050,"banner":10050,"bio":10265,"body":10266,"description":5496,"extension":10049,"meta":10270,"name":10264,"navigation":10052,"path":10271,"seo":10272,"sitemap":10273,"social":10274,"stem":10275,"__hash__":10276},"authors\u002Fauthors\u002F法法.md","法法","123",{"type":8,"value":10267,"toc":10268},[],{"title":5496,"searchDepth":10026,"depth":10026,"links":10269},[],{},"\u002Fauthors\u002F法法",{"description":5496},{"loc":10271},{"github":5496},"authors\u002F法法","o5pdVuPCfTmhkDCpvgy4YmAP0CGdvFluPvjhgvQVbsI",{"id":10278,"title":10279,"avatar":10280,"banner":10050,"bio":10281,"body":10282,"description":5496,"extension":10049,"meta":10286,"name":10279,"navigation":10052,"path":10287,"seo":10288,"sitemap":10289,"social":10290,"stem":10292,"__hash__":10293},"authors\u002Fauthors\u002F灰海獅.md","灰海獅","\u002Fimages\u002Fuploads\u002Fimg_3279.jpeg","守夜人",{"type":8,"value":10283,"toc":10284},[],{"title":5496,"searchDepth":10026,"depth":10026,"links":10285},[],{},"\u002Fauthors\u002F灰海獅",{"description":5496},{"loc":10287},{"github":10291},"https:\u002F\u002Fgithub.com\u002Fyuiri333","authors\u002F灰海獅","iZoSIFbQdS-6v3LiK1txgxnIMKy-d2CyZXQk9CMua_s",{"id":10295,"title":10296,"avatar":10297,"banner":10298,"bio":10299,"body":10300,"description":5496,"extension":10049,"meta":10304,"name":10296,"navigation":10052,"path":10305,"seo":10306,"sitemap":10307,"social":10308,"stem":10310,"__hash__":10311},"authors\u002Fauthors\u002F花夜.md","花夜","\u002Fimages\u002Fuploads\u002F1772719470518-791_20260218161129.png","\u002Fimages\u002Fuploads\u002Fimg_2446.png","無論你身在何處，我都會在這裡等你",{"type":8,"value":10301,"toc":10302},[],{"title":5496,"searchDepth":10026,"depth":10026,"links":10303},[],{},"\u002Fauthors\u002F花夜",{"description":5496},{"loc":10305},{"github":10309,"twitter":5496},"https:\u002F\u002Fgithub.com\u002Fflowernight0709","authors\u002F花夜","a7jeQiF_JkawgYIR-aYSGceJdDP6Z-OWydsICvgSIzs",{"id":10313,"title":10314,"avatar":10315,"banner":10316,"bio":10317,"body":10318,"description":10322,"extension":10049,"meta":10325,"name":10314,"navigation":10052,"path":10326,"seo":10327,"sitemap":10328,"social":10329,"stem":10332,"__hash__":10333},"authors\u002Fauthors\u002F輝月.md","輝月","\u002Fimages\u002Fuploads\u002Ffb_img_1771085634823.jpg","\u002Fimages\u002Fuploads\u002Fimg_1751.jpg","天下布魔好好玩",{"type":8,"value":10319,"toc":10323},[10320],[16,10321,10322],{},"準大學生，目前正在製作TFR模組",{"title":5496,"searchDepth":10026,"depth":10026,"links":10324},[],{},"\u002Fauthors\u002F輝月",{"description":10322},{"loc":10326},{"twitter":10330,"github":10331},"https:\u002F\u002Fx.com\u002Fhuiyue945","https:\u002F\u002Fgithub.com\u002Fhuiyueyea","authors\u002F輝月","koUocBihphDy3453-nAcolM7JJYwI7UMBpVkf1JQrMQ",{"id":10335,"title":10336,"avatar":10337,"banner":10050,"bio":10338,"body":10339,"description":10343,"extension":10049,"meta":10356,"name":10336,"navigation":10052,"path":10357,"seo":10358,"sitemap":10359,"social":10360,"stem":10362,"__hash__":10363},"authors\u002Fauthors\u002F阿西狄亞.md","阿西狄亞","\u002Fimages\u002Fuploads\u002Fimg_20251215_121849_589.jpg","君は実に馬鹿だな",{"type":8,"value":10340,"toc":10354},[10341,10344],[16,10342,10343],{},"我是阿西狄亞，阿西狄亞的阿，阿西狄亞的西，阿西狄亞的狄，阿西狄亞的亞，你可以叫我阿西。",[16,10345,10346,10349,10350,10353],{},[20,10347,10348],{},"我說的所有事情都抱有極度主觀的看法以及意見","，如果你有其他想法，",[20,10351,10352],{},"你是對的","。",{"title":5496,"searchDepth":10026,"depth":10026,"links":10355},[],{},"\u002Fauthors\u002F阿西狄亞",{"description":10343},{"loc":10357},{"github":10361},"https:\u002F\u002Fgithub.com\u002FAcedia0130","authors\u002F阿西狄亞","q5ECEDl8-0Y33tPck0lYZnzPjFdJkrOnBN7HkAO3pls",[],[10366,10375,10384,10393,10402,10411,10420,10429,10438,10449],{"id":10171,"title":10172,"avatar":10173,"banner":10050,"bio":10174,"body":10367,"description":5496,"extension":10049,"meta":10371,"name":10172,"navigation":10052,"path":10180,"seo":10372,"sitemap":10373,"social":10374,"stem":10187,"__hash__":10188},{"type":8,"value":10368,"toc":10369},[],{"title":5496,"searchDepth":10026,"depth":10026,"links":10370},[],{},{"description":5496},{"loc":10180},{"website":10184,"twitter":10185,"github":10186},{"id":10190,"title":10191,"avatar":10192,"banner":10193,"bio":10194,"body":10376,"description":5496,"extension":10049,"meta":10380,"name":10200,"navigation":10052,"path":10201,"seo":10381,"sitemap":10382,"social":10383,"stem":10207,"__hash__":10208},{"type":8,"value":10377,"toc":10378},[],{"title":5496,"searchDepth":10026,"depth":10026,"links":10379},[],{},{"description":5496},{"loc":10201},{"github":10205,"twitter":5496,"website":10206},{"id":10210,"title":10211,"avatar":10212,"banner":10213,"bio":10214,"body":10385,"description":5496,"extension":10049,"meta":10389,"name":10220,"navigation":10052,"path":10221,"seo":10390,"sitemap":10391,"social":10392,"stem":10226,"__hash__":10227},{"type":8,"value":10386,"toc":10387},[],{"title":5496,"searchDepth":10026,"depth":10026,"links":10388},[],{},{"description":5496},{"loc":10221},{"github":10225,"twitter":5496},{"id":10229,"title":10230,"avatar":10231,"banner":10050,"bio":10232,"body":10394,"description":5496,"extension":10049,"meta":10398,"name":10230,"navigation":10052,"path":10238,"seo":10399,"sitemap":10400,"social":10401,"stem":10243,"__hash__":10244},{"type":8,"value":10395,"toc":10396},[],{"title":5496,"searchDepth":10026,"depth":10026,"links":10397},[],{},{"description":5496},{"loc":10238},{"github":10242},{"id":10246,"title":10247,"avatar":10248,"banner":10050,"bio":10249,"body":10403,"description":5496,"extension":10049,"meta":10407,"name":10247,"navigation":10052,"path":10255,"seo":10408,"sitemap":10409,"social":10410,"stem":10260,"__hash__":10261},{"type":8,"value":10404,"toc":10405},[],{"title":5496,"searchDepth":10026,"depth":10026,"links":10406},[],{},{"description":5496},{"loc":10255},{"github":10259},{"id":10263,"title":10264,"avatar":10050,"banner":10050,"bio":10265,"body":10412,"description":5496,"extension":10049,"meta":10416,"name":10264,"navigation":10052,"path":10271,"seo":10417,"sitemap":10418,"social":10419,"stem":10275,"__hash__":10276},{"type":8,"value":10413,"toc":10414},[],{"title":5496,"searchDepth":10026,"depth":10026,"links":10415},[],{},{"description":5496},{"loc":10271},{"github":5496},{"id":10278,"title":10279,"avatar":10280,"banner":10050,"bio":10281,"body":10421,"description":5496,"extension":10049,"meta":10425,"name":10279,"navigation":10052,"path":10287,"seo":10426,"sitemap":10427,"social":10428,"stem":10292,"__hash__":10293},{"type":8,"value":10422,"toc":10423},[],{"title":5496,"searchDepth":10026,"depth":10026,"links":10424},[],{},{"description":5496},{"loc":10287},{"github":10291},{"id":10295,"title":10296,"avatar":10297,"banner":10298,"bio":10299,"body":10430,"description":5496,"extension":10049,"meta":10434,"name":10296,"navigation":10052,"path":10305,"seo":10435,"sitemap":10436,"social":10437,"stem":10310,"__hash__":10311},{"type":8,"value":10431,"toc":10432},[],{"title":5496,"searchDepth":10026,"depth":10026,"links":10433},[],{},{"description":5496},{"loc":10305},{"github":10309,"twitter":5496},{"id":10313,"title":10314,"avatar":10315,"banner":10316,"bio":10317,"body":10439,"description":10322,"extension":10049,"meta":10445,"name":10314,"navigation":10052,"path":10326,"seo":10446,"sitemap":10447,"social":10448,"stem":10332,"__hash__":10333},{"type":8,"value":10440,"toc":10443},[10441],[16,10442,10322],{},{"title":5496,"searchDepth":10026,"depth":10026,"links":10444},[],{},{"description":10322},{"loc":10326},{"twitter":10330,"github":10331},{"id":10335,"title":10336,"avatar":10337,"banner":10050,"bio":10338,"body":10450,"description":10343,"extension":10049,"meta":10462,"name":10336,"navigation":10052,"path":10357,"seo":10463,"sitemap":10464,"social":10465,"stem":10362,"__hash__":10363},{"type":8,"value":10451,"toc":10460},[10452,10454],[16,10453,10343],{},[16,10455,10456,10349,10458,10353],{},[20,10457,10348],{},[20,10459,10352],{},{"title":5496,"searchDepth":10026,"depth":10026,"links":10461},[],{},{"description":10343},{"loc":10357},{"github":10361},1777302338145]