Robustly improving LLM fairness in realistic settings via interpretability

%PDF-1.5 %???? 1 0 obj << /Metadata 3 0 R /Names 4 0 R /OpenAction 5 0 R /Outlines 6 0 R /PageMode /UseOutlines /Pages 7 0 R /Type /Catalog >> endobj 2 0 obj << /Author (Adam Karvonen; Samuel Marks) /Creator (arXiv GenPDF \(tex2pdf:\)) /DOI (https://doi.org/10.48550/arXiv.2506.10922) /License (http://creativecommons.org/licenses/by/4.0/) /PTEX.Fullbanner (This is pdfTeX, Version 3.141592653-2.6-1.40.25 \(TeX Live 2023\) kpathsea version 6.3.5) /Producer (pikepdf 8.15.1) /Title (Robustly Improving LLM Fairness in Realistic Settings via Interpretability) /Trapped /False /arXivID (https://arxiv.org/abs/2506.10922v1) >> endobj 3 0 obj << /Subtype /XML /Type /Metadata /Length 1640 >> stream <?xpacket begin="" id="W5M0MpCehiHzreSzNTczkc9d"?>

Robustly Improving LLM Fairness in Realistic Settings via Interpretability

Adam Karvonen

Samuel Marks

http://creativecommons.org/licenses/by/4.0/

cs.LG

cs.AI

cs.CL

<?xpacket end="w"?> endstream endobj 4 0 obj << /Dests 8 0 R >> endobj 5 0 obj << /D [ 9 0 R /Fit ] /S /GoTo >> endobj 6 0 obj << /Count 16 /First 10 0 R /Last 11 0 R /Type /Outlines >> endobj 7 0 obj << /Count 31 /Kids [ 12 0 R 13 0 R 14 0 R 15 0 R 16 0 R 17 0 R ] /Type /Pages >> endobj 8 0 obj << /Kids [ 18 0 R 19 0 R 20 0 R 21 0 R ] /Limits [ (Doc-Start) (table.caption.9) ] >> endobj 9 0 obj << /Annots [ 22 0 R 23 0 R 24 0 R 25 0 R 26 0 R 27 0 R 28 0 R 29 0 R 30 0 R 31 0 R 32 0 R 33 0 R ] /Contents [ 34 0 R 35 0 R ] /MediaBox [ 0 0 612 792 ] /Parent 12 0 R /Resources 36 0 R /Type /Page >> endobj 10 0 obj << /A 37 0 R /Next 38 0 R /Parent 6 0 R /Title 39 0 R >> endobj 11 0 obj << /A 40 0 R /Count -3 /First 41 0 R /Last 42 0 R /Parent 6 0 R /Prev 43 0 R /Title 44 0 R >> endobj 12 0 obj << /Count 6 /Kids [ 9 0 R 45 0 R 46 0 R 47 0 R 48 0 R 49 0 R ] /Parent 7 0 R /Type /Pages >> endobj 13 0 obj << /Count 6 /Kids [ 50 0 R 51 0 R 52 0 R 53 0 R 54 0 R 55 0 R ] /Parent 7 0 R /Type /Pages >> endobj 14 0 obj << /Count 6 /Kids [ 56 0 R 57 0 R 58 0 R 59 0 R 60 0 R 61 0 R ] /Parent 7 0 R /Type /Pages >> endobj 15 0 obj << /Count 6 /Kids [ 62 0 R 63 0 R 64 0 R 65 0 R 66 0 R 67 0 R ] /Parent 7 0 R /Type /Pages >> endobj 16 0 obj << /Count 6 /Kids [ 68 0 R 69 0 R 70 0 R 71 0 R 72 0 R 73 0 R ] /Parent 7 0 R /Type /Pages >> endobj 17 0 obj << /Count 1 /Kids [ 74 0 R ] /Parent 7 0 R /Type /Pages >> endobj 18 0 obj << /Kids [ 75 0 R 76 0 R 77 0 R 78 0 R 79 0 R 80 0 R ] /Limits [ (Doc-Start) (cite.bau2018identifyingcontrollingimportantneurons) ] >> endobj 19 0 obj << /Kids [ 81 0 R 82 0 R 83 0 R 84 0 R 85 0 R 86 0 R ] /Limits [ (cite.belrose2025leaceperfectlinearconcept) (figure.caption.4) ] >> endobj 20 0 obj << /Kids [ 87 0 R 88 0 R 89 0 R 90 0 R 91 0 R 92 0 R ] /Limits [ (figure.caption.5) (section.1) ] >> endobj 21 0 obj << /Kids [ 93 0 R 94 0 R 95 0 R 96 0 R 97 0 R 98 0 R ] /Limits [ (section.2) (table.caption.9) ] >> endobj 22 0 obj << /A << /D (Hfootnote.1) /S /GoTo >> /Border [ 0 0 1 ] /C [ 1 0 0 ] /H /I /Rect [ 496.528 214.223 502.506 226.743 ] /Subtype /Link /Type /Annot >> endobj 23 0 obj << /A << /D (Hfootnote.2) /S /GoTo >> /Border [ 0 0 1 ] /C [ 1 0 0 ] /H /I /Rect [ 341.458 192.305 347.435 204.826 ] /Subtype /Link /Type /Annot >> endobj 24 0 obj << /A << /D (cite.tamkin2023evaluatingmitigatingdiscriminationlanguage) /S /GoTo >> /Border [ 0 0 1 ] /C [ 0 1 0 ] /H /I /Rect [ 472.807 109.615 504.996 120.559 ] /Subtype /Link /Type /Annot >> endobj 25 0 obj << /A << /D (cite.tamkin2023evaluatingmitigatingdiscriminationlanguage) /S /GoTo >> /Border [ 0 0 1 ] /C [ 0 1 0 ] /H /I /Rect [ 107.004 98.656 129.123 109.6 ] /Subtype /Link /Type /Annot >> endobj 26 0 obj << /A << /D (cite.tamkin2023evaluatingmitigatingdiscriminationlanguage) /S /GoTo >> /Border [ 0 0 1 ] /C [ 0 1 0 ] /H /I /Rect [ 132.872 98.656 154.79 109.6 ] /Subtype /Link /Type /Annot >> endobj 27 0 obj << /A << /D (cite.veldanda2023investigating) /S /GoTo >> /Border [ 0 0 1 ] /C [ 0 1 0 ] /H /I /Rect [ 158.817 98.656 221.258 109.6 ] /Subtype /Link /Type /Annot >> endobj 28 0 obj << /A << /D (cite.veldanda2023investigating) /S /GoTo >> /Border [ 0 0 1 ] /C [ 0 1 0 ] /H /I /Rect [ 225.007 98.656 246.925 109.6 ] /Subtype /Link /Type /Annot >> endobj 29 0 obj << /A << /S /URI /Type /Action /URI (https://techcrunch.com/2025/02/20/mercor-an-ai-recruiting-startup-founded-by-21-year-olds-raises-100m-at-2b-valuation/) >> /Border [ 0 0 1 ] /C [ 0 1 1 ] /H /I /Rect [ 178.752 69.028 259.039 81.895 ] /Subtype /Link /Type /Annot >> endobj 30 0 obj << /A << /S /URI /Type /Action /URI (https://www.aztechcouncil.org/paradox-software-firm-hits-unicorn-status-with-new-200m-capital-raise/) >> /Border [ 0 0 1 ] /C [ 0 1 1 ] /H /I /Rect [ 261.78 69.028 363.362 81.895 ] /Subtype /Link /Type /Annot >> endobj 31 0 obj << /A << /S /URI /Type /Action /URI (https://www.reworked.co/employee-experience/linkedin-joins-the-ai-agent-trend-with-launch-of-hiring-assistant/) >> /Border [ 0 0 1 ] /C [ 0 1 1 ] /H /I /Rect [ 123.143 58.153 227.226 71.02 ] /Subtype /Link /Type /Annot >> endobj 32 0 obj << /A << /S /URI /Type /Action /URI (https://www.indeed.com/press/releases/indeed-launches-ai-powered-smart-sourcing-to-make-hiring-faster-by-matching-and-connecting-people-with-relevant-jobs) >> /Border [ 0 0 1 ] /C [ 0 1 1 ] /H /I /Rect [ 229.968 58.153 336.767 71.02 ] /Subtype /Link /Type /Annot >> endobj 33 0 obj << /A << /S /URI /URI (https://arxiv.org/abs/2506.10922v1) >> /BS << /W 0 >> /NM (fitz-L0) /Rect [ 12 223.23999 32 568.76 ] /Subtype /Link >> endobj 34 0 obj << /Filter /FlateDecode /Length 140 >> stream x?EJ? ?0??W??nf??? ??͒?x?5??(???? 23̃?? ;n?c?othύ?S?Y???j?{n?t??~?7????~\??C? 5 ??w????cШ?~??N?R???!ZȵA'?򐶴N??7?n' endstream endobj 35 0 obj << /Filter /FlateDecode /Length 3472 >> stream xڍZ[?۶~?_????:?bI???؎?u?ڛ?I?$Vb͋‹7?_?o0???/"8 ???P?f??6???o???2?7???*???o|/?$Q?Ɓ??-7?8o{}?v??v??Ջ۫߯|l?6?,Unƛ????7oSb?ۍ?Y?y0˚M??x֛?W????\??|?q??B??8K?8c??ټ.p?Df??&*???.V??v?Ώf??????wn??????tZ?pXj??^;(v?,?+??o???N):??????????Z;. ?(T? ?????p^?Q '?|?g??ޙ??/?k??d?L?%`?[??Vnp:??Ȧ?,H?o??k??????ov>?ME????d??{???9??? τ?l??T????|????X0|?????,s?T?,?+X-6?+?7???=+????.﷾?|?????L? ??????R5~?j?8ZY,ƕ? ?C?$???7yU?E׬???ȍ?l?|7???y3? }?????N|֎??;Vʼn?@A.Ċ?wU?mv?y??V?OH?8?(]5???'???*?"ud????????0p?(?`??(? u?Z?N???)??i?R????E??͛O?!y/˪??u>T??~?)???H????kx?P??a??Ь,NxO??D??c]?Xu??t???4?Z????@s??8?!e??u1r?V5Ǽ?????6???!.?{A??%C?j?"R???u?[#??u??B?χ?&vĉ????4L?a?????a???0?h? )??????|`ȱ??(?????h?6e[?W?j?}?PA?6r)?9!DNѵc??5'v4_?y=????yм?W?S??zpS??~?m??{?,??ب/?????0*}e(?????)??;??f??lTDz9???,r~ަ?$ޒ?e??b???4?z??i?8W99?%b??:,?%?w??4s?y?"?T???mF?h??????????@u?2^??c_?m??Өy%+???J?R?x0??C6XB᫓Xg??Ԭ8Tzy<???6z?w?I???:ᱻ?n????CG?Q*??b]???Af???L??H9??K?N@!?6???2" ?<Ы???L9??{???u>????xߵ?e????T܈?/?|8G??r??w3????B3??c&F7M?S??*?~?S?-A???C?;^Ӊ??A[4K???U?J˖hi6*??f???TxʨkD???ҚG?h?FOe?T?S????C?WG1e?{7?s???vI??=U:G????8?Z??Nߌq?l[{??w???Tе???,??>?m?˪?C?]d???swds??'?r(#@ՒG?e??mE?z:?f??b??????^?xY퓜4vr?z˜??q?{??'V?.???G6?2?t?D6?'BR?3yW3c?]??<>ƺ?ߍC?kX? 3|?:???-dgI^?#??XT?u^|?aG(X????&H!m\???ٺ0??M^ˢ3|r????;????wh??R9m?>????ȥ?t?>??+[??)??̽!?R??VV? 1?Ӱ=? ?Ln?j???n4?????/t??8U8Em"r/?ܨik]-ҤЄ^bU?Y5??? ?M*?)? n??ԝ%b???c?O??U?p$7??)O??d?f??u???@?w?e?>vD?/CB???????E?B?(???-|Pi??c?$lNϓ????"??$?qb??Ep ?Gl?X$??]??:7*Hk>??FQT??֜@/? [???:????p?O???G?u?m????p46H?i?w.(3???hP^?Y?????? S0I??h??????$X^i????2J3??????#G0?K?????<??7??`!?K??G~C1"NK?^?is?Q???Ȟ??;??o??->?([??{HI???????+?׶?b?5??7$$???=?????6??w~?z^? 6 ?~*2LP0?P@5 ?$??kܜ+6??N??b?b ?? ? ?K??eg5&?i??