Neural networks that learn non-linearity without activation functions [pdf]

%PDF-1.5 %???? 528 0 obj << /Linearized 1 /L 9636945 /H [ 3765 863 ] /O 532 /E 60950 /N 53 /T 9633505 >> endobj 529 0 obj << /Type /XRef /Length 148 /Filter /FlateDecode /DecodeParms << /Columns 4 /Predictor 12 >> /W [ 1 2 1 ] /Index [ 528 442 ] /Info 157 0 R /Root 530 0 R /Size 970 /Prev 9633506 /ID [] >> stream x?cbd?g`b`8 $?b@?F ?b5 ?# V??$?D??) ??H0& ?+ ?Z??P ??????? ??}@?4 !?${ƕ)aq?@?t&Fa?Q???p!??F ? ???Q???܁v?(A?h\6?.?> U?? endstream endobj 530 0 obj << /Names 969 0 R /OpenAction 924 0 R /Outlines 835 0 R /PageMode /UseOutlines /Pages 834 0 R /Type /Catalog >> endobj 531 0 obj << /Filter /FlateDecode /S 1133 /O 1262 /Length 773 >> stream x?c```b`????????? ? `6+?ɲ$ Ƞ?? ??$X`?@i? ??sb ?10?p`~?????????(????&.??S`?=????3???c????p4H`?{?Ͳ?'?IDŽA?D?k?AƆ?9X?6p?f?`??????V??;???/?k?3)*m@?;`?:????-????d?F ??F???=?Y???6{?ʉ?os???d???^t??㆜?? "?]:?????gt?Xt???W?+aYY?n88 ???????+??xR??-K??kڲպ?y?J&?ٚ???7?N?+??t{?a??)??ߐ?08??̅(?)??mp??????6????r????:7!?E&FsS2??$ǯ????i9?b??'ȅ'!ܟ+V????3????e֬5??H??|?h?j??tn!l?\??fmH?j1? &0L??0??? p)?-9?? ?$"b,?xB?h?-E?&?o?0YF?p?/? Y?#???=8???fo?"? ??? ?3?????????F???.????????)??????vo??׽1???w???!?k??j?ù?_{A&?z??ǭ??5Pb?zIr?vo8?y?7???lb??d?%??\v?2?3D??a??`?(???????t8[??3(e?o`VY0?k7??x???3?>``di?f?????q??????a?*{4/7k?|?y+?~?q????K,&-`?? *??? ?e?@ endstream endobj 532 0 obj << /Annots 926 0 R /Contents 533 0 R /MediaBox [ 0 0 612 792 ] /Parent 754 0 R /Resources 579 0 R /Type /Page >> endobj 533 0 obj << /Filter /FlateDecode /Length 2937 >> stream x???n$?1?W???? ???c?2Û.??N??#???*V??fό?A????T?_??oo?}?巺?K?[R?m?B??|?????????Em6???`?ϋ?????oS ???}?{?q3?;w??EHn?^~?????????5????ݧPV]???jU????v+?۝v???KJŌ?}?a?]߳I??~???L?????US?z] ??;?]?ƺW1?t????W[G??+???j?p????f??3?K?]??Hb ? #xL%? ?ve?RwA2)??+y?-{S]???ڎ\:|?#??:?%0?_?:^? K??阈?*G?FArG??d?PL? ??@H?ʄK*??M?sz*ϸb@? C????O????0???z?`?"nD?`,d?7??Y???th?Qd?r_i?:7?\?KD??VW?T??g??8?A?????OȽ?????=?????؜??d?n?? ???????*)???>?Pys?H ??W?M+ J??o??Vv?5 AD4fa @T!eG?tC? ?;}r??????>?,ϳ??OP?N4x??o?U">PCO3M????`?F}MV??x??? ?Z*?x??{t??? ?????n?2??R);Ì? ?Q0?????SO?Q?}??Q,??x</z???>?V\??⇸???6AZ?$?(C?`T??N?h,?B#??-??V??????7?????(ٙm?P?#??X&??Z[ ?@i?\{?:?V)??'??R?|??S@?X1??#Vy?>b???d???"l??Қ??^$?1?|ܹ'F???pri:??'դ????r,_[??? ??ޓ1?'[??w%?mL?x??r`,? ?p?f??f7.???x?ͱ?*Fb?þ?*%Gzj^q??B)?;WD-|#=?ò???"???F?o?Q4??T?~?p2??]9?N?QXCb=?o????+???T??72??2xsϥ?]???$a?GQ?W??߹ ???[^?f'E贴??gr???Q9?$?9?A??/??p?|??/?!H??????=? ?;q?h?\8S樄7Trr?R="?^?R?xt)c????i?"G_???"??U???????*???X?-?o?c???mR';?;Q???R??J?:??F?QKd??K?Q?6K??z_5?????q?Ҷ%???2????ST???C??Yʭ𽔺?[2???m?5?¨??1??D??n?-|?Q.M?i^?wHd?n???9s??w?ǣ?nw?"?."g?#???H? ???J4흄c???Y4RzK???~????yr?CT?]x?K4T??s?T??4AD@?4??U?I??`#j?H?I`&??????|q(?W ???pZ?* N?bf7ts?????tcv?U?G?ҩ$J>k???)? ܲl'?d?TM?t?YH???????M?Ma?.?r{??J{? 헺b$??$?k?ș???????I?J# ?6ũ?-Q?f??L ???m??O???>?i!?޸cl}huh?#a??ûA??`?1\?J8?b??????qI&҇l1M?????q?:? ?????X?F(??K?U?)?@??1?h_ f?a????_C?????[k?:?l,??k?]????t?a6]?S?2??V??y?-"?R??mʱ?͂L?\c[O???_? ???*3Yӓ!Q٩???u`?h??Ե e??D?_:?H??&W?~??Y? n?[??pfSy?'Cya?h?V?x?1?????m??i?P?? ??IQ ?;??%5?L)Yܽ{?iv??ת?/?FZ?6?E???au?u?????R?J?? ?3?T????T+?;?r?? ث?X??NJ+K?ЪvO?]??}t?Ya??&??????Qo?7i?7? x+?b??o!????˛?? ?$??@ē??p???U?????\????W?Y`2Su? ?~????%??z? ")????/??5?29G^?jE??????????8d`?Ԇ??D?3? ???g??/???g놯?wn??CUE: i3aAse??ew???????s??k?VIër?????3??h????T?z.?ZT;Q???S?Bbd5??j HvO???kw???3??l?4졲?i?T,?#$v?%?4???t??!Ƣ(N6?^}ZR??????&?۬?? ? @???q8??3q߻C??zF ?2????2?Y_?6???j??~ѬsK?sY??????![?LW.???7??b?C???Hk6'?x?KZ???????ϵ?q?/p?8!?ǀ$P?t??FkuL?3???f???m*?"U??????]OҐ?7?????Z?65?: ?z?!?F?2d2??z?aZsx????Z??·?????Q^?E??4????梑?.?N???i??>??d^]z?댨? h??6?-??U?_?d'uxxuH?,????????G??+(7̄?0??$Kqy?'?g?7??????@?? ?i"܀?-s???%??k?$?R??19U??s??&p4E?V?q6????ӕ?/??7J??9Q??????c??x????$???>??ՠ??Q?F??k,5????C??N^m?Q??&'??Tql?ldن=??C?Ӏ1?z?֑?lN?:???B?'\Oߞ=???M?p?_?0??F?#E??mi?6?/:?-ezI?fщ?-7???WF? endstream endobj 534 0 obj << /Filter /FlateDecode /Length 23 >> stream x?c``?pT`T?????? ?; endstream endobj 535 0 obj << /Filter /FlateDecode /Subtype /CIDFontType0C /Length 2823 >> stream x??WTW?&E?Ԫ??3???VV?}???B?j??? $?"Ix?I?3H +Z???k?ۭ???cm???t?;?r??D?n??v?ݝ??$?????}????? ???? Oܹ????+?<?C ?+?{t?&?0G???,e??|?Y?0ws"?!?L\?qS????|? ?xZG???7?A8?;?kw?9?e????偎?$$? ???S?N'?3?#?᧯???~0Vz?D&?+?5??kW?G??Hj?0-[Z$ϖP¼tjs$?l$?UZ??J???yT?H,?ɠ?T?hU ??T?LZp@~$? ?ȩ"?,?b?e??P.J? ??E2J!Qq?? ?Fi???"I??E???%?(?Bq`]T?? 3R*ˌ?`?ȣrf&ɣ??޸mk??-?b7l???(VPR?.R%9?????[??\a?oӥ??M?<?DQ???d)??B? O"1H?4?yق<?lGv!?H "DR R?e1e?0$,?n0??pN???5/nF??[??? Y?b?%???l???9^?z?ʴV??>??K\f ?#?-?ueF]??ؤ,ٙ?k|NK???|???z?l??ݥ?UR-Ѭ??ۍ?`8F???&?=??D&?ˆ?c-GkHo???$?-O$? ?7] ?3@??q??Q???aM5mԫ?Щ?J@}h9`U?g????)?%?[E??hM)?|??O?py??P? ??V0??5p?.ԮUi?m.???&_v??0Y,Xm]J?yNj????0·\?????L?:??????U??DϙcM??=8?1?6?KH?T?a1??ǯ?;Z?? ?j?[\? ?U1?7??/]????u?:???&C??l?2??e ????v-^?kȘ? ??s&??o?]$ǟ>?׎???j?H???9???ѱ??8???e0b???7?p???6w c??. K????)?Rg2[?:?Yi ?Q?eF?Q?????1?J?? ???X???? ??}?a??ׁq?G??S?y???n?H>???ɚ?>?R??lT4X 7??W6?7??,?k??x???z?hKn.?=`>>_??O\??Z"S%]??W????v״???C?{{5mpCB?,CL\?QK????8ѴFWF??ӗQ?A3??]????D?mv u??E?恮?????E|????=?V&?Ym!?8 ???p??????%@惰?C? ???7???~???67??X?lsw5a/Hz?Ve?uE-m?7?j?D???_??.?!_?)R?#ő??hU?Z^??/?Z?LAgA?"G"͔????{;??ЩE????L- A??r??TKDK?0>???) ?EE?h??? K?!????zw?ݴ׭2k??$}Rj?% ז?%?n??,?Q??EB|z??????ʙA??㵚L^?&??:?Jc??S雚?¹q????Q??|?/??o??kZ????e??ޔӣ??MN??Ww΂?~???-?ԱV????k?? Ӄ?9?5?x`w?A????? ޜ??ʜG??FS>?x?????iW?3???ź ???V?$?z-?݇?~?5Tom?IC&??[????[???&????h?٤i?=6??N2s?i?R???ڡ?!?-?G?}?????6ze?x?? ????7?7??g?R:?? ??? ɓj?/ϒ?J?R??x?n??6????N????)??ud,?%?j??^??5??#Z??B0;|??wt??v?]ޢ?%?\}9??~x????d{?????)뷤}=?F`?ר??,}v?ؒ?W?aW?6:?1??? ?? O?fԫ1? pȟ??E????*?ֳFMw?B?2 ??-???1Y???|???A??y?????z?3?0Kn?͟?I????|?:?C????q?]W?e.???O??! .??p)?7?l? ??????!/i#?'{_?'?_l???y?g?{?z83?d-???\PxQЦl?f?Jr???}]?],?*??}`??a????????? җ?zbkq?4WT֏x??f? ?;+???J???Ϯ|?]}d?#`?????sK4?G,?C??[???y+[?x? P??]O|v??JWF?ū2`˿?5]??&?gR??O&g????$??M k5???=X??$dt?e6? ?߰X.? ?W????+?ז?--S"'F+???????bԣ?Qp?????.k!sU?? ??S?????g?|??????'????1P.???%CD??Ķ?5?????!?u}#ػW????b?F?^A?h=pP??jj??@??,la?Ѵ?Bc4????;??Y?u?q8?-???e????H07m?-??V=?j͓?{?j??Z? ?Uњr??DG?)?QoPј?V]?0[?????n??l>?N?w?6???y?-?&_`a?:??q? ZЩ?(j!?;?$gH?#?1?y^???B?"24t??????-???iC?pVrd??Wvi?1ըj'?'r??c?Ę???C {?6Y???S??Ճ t???7X-????"?_?2A`???J????? &??+????Lt???x????XA}?j>PY???0}??4??=g?R???/N?K\5????j?y??|ʄ?VO??]eJ?Z?!b???Jڳ?]}?Щ??z?&?mVw?O???\M1?ީJMƖ^??i:l??#??Y1 ?????W?c6??\Z[?̪??ܰ?%t ?????<????_?y???2?n?05??? 9Q?Y4?o????l2??&?@????bjb??f2??6?ܿ?gա endstream endobj 536 0 obj << /Filter /FlateDecode /Length 458 >> stream x?}?Mo?0????C??@1$!M!??h??J??8{??ld????Ǵ?*?H ޙ???0???>@4?et???x?[2?l4P?? 1??=}3?Ӣ?TJ?76?R?<??=?I??????g??????+????M?VQ??g?#????????Q+??"?E???V?4?e??OJ?q??H,?x4}?J????k??TH??O??>??ݥ롩?Q????[+v??x?7$~5?T':?nц솶=??C?s*?h+???? ????V???M?L?_Z????r-?kk?V' k???ui?????GD0???(?ߵq???b,KrGi?iΐQ?!?HsO?i??;O󠭐?X?@?C-?j??^ t6Bڠ??@?Y??5Ӡ??E?)c???0o?;???2? ??̐V9???˵???????{?~??ٹS? >f?խ?????q?Z??nh7 endstream endobj 537 0 obj << /Type /ObjStm /Length 2400 /Filter /FlateDecode /N 93 /First 822 >> stream x??Z?o7~????w/?!9$Pȥ???"???6??Z??˻?Jj???߬V?:RY???0?] ?o????Q!$?#??2??Q? ?????c??l?W.e,? ?? h??I??X: ?A???E?\?$,?V> ?f??B&?I?cR?0?ț?<???(p"c֐?o0a4X?QB#)?pVV??x?p$Q@G?1\f??F,?k????g???ؑH?d???S??s0TE?F??8?d??_S???X:I???lVl???b"b=???ʊ7?Nt?,o?bvB'?-:V??Tr???*???g?????J???TJRh?Jy????@?l??at?m?$????e??ɂ?;tX???????wc??K? (?5r?????#?9????~?A?~Rﰁ????? ?6???v?X????wԴOKN[??Q??4?>mأ??h????&iUا??y-??C?m֖???ys??y- ?!ZH???.?m?b???sN?R?!??7?Z?ݲn/?j????~^߭???????ٛ??????]Y?X???b????A????j?K=_?V;։E????K??T%????????]?f?o?5?u?j?nu??|?o???6{ZtWͼZ? ?M??9!??????к??b?,???/A?f?Y?BGluL??OB8???ۛN Vww}????VM?Ns??ڡL???????5??/؜xY?w%??????ar?????yQ??-??Bhްn ??s??jz 3=Vh>4?e ???(??1E|B?i7?6x?S4?n/?????GW?z?l?????_4m]??Y-???!?Y???? 29??9 ?͢???ֵp????C ?赑 eD??\? ?[??8???` ~?V??{1895? J??u????j^C??x??Wo?W?J^?ͫ??] ?s?m?0??3??ы?I??pj|@|??@????????H??-?mݷXj??m???k???۟? ????,?y????c~??C???0?pH?p?.??????Wuw[??f^$?p[??Bz¤%j??????`tv?Q??F?mګ???,2$??? t?? »I^T ?Z?K??????ȵyy???? ]8 $?:??E?-?d??ަ??p𲯮???S?]7??u?A???$????杆?F䯷?rU?w!dG?D@b??: ??7)??=???_???ȅKp??P???;??p????n??n/,?P?+&-e??d??m ??FB??V?:??/?"X??Z?R`j휩z ??MYJ?a?'Ą,??񪺮????J?Y???0???l)?q?j: ??67g??MVKi?`???mv?#&y@zė ѯ6?̚()????f????+PQ@???a????L?Q??B??1??R??q[?q$c??!)?l?