Repository navigation
State the conventions corpus's parts, and score them like CLDR's - #692
Merged
Merged
Conversation
Follow-up to #686 (#691). 30 of conventions.py's 35 rows now carry `fields` in CLDR's vocabulary, drawn from the UK naming-practices guide and Cultural Atlas research the rows were written from; build_conventions_corpus.py writes them to conventions_fields.jsonl and cldr_agreement.py scores them as a fourth source, recorded in conventions_agreement.json: 5 of 30 agree in every role. Derek's calls: a word of address written after the name (Bey, ssi, san) is a new `honorific` field scored inside `suffix`, as nameparser reads さん; Kaur is given2, Péterné the given name, H. Budi Santoso a title and one personal name, Ma.'s Cruz a surname2 and Vda. de Mora part of the surname. Five rows stay unstated, since a stated field reads an unstated part as an empty role: the kunya, the glued -san, Szabóné's married form, and arap/ole, patronymic or family name by family. The fields file is therefore a subset of its corpus, which test_cldr_agreement now allows for this source alone. The other sources' records do not move; the corpus file is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #692 +/- ##
=======================================
Coverage 99.03% 99.03%
=======================================
Files 46 46
Lines 4541 4541
=======================================
Hits 4497 4497
Misses 44 44 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
…the text Review of #692: - A field name no rule reads now raises in expected_roles, for every source: a mistyped honorfic was expected in no role and passed as agreement. CLDR's locale and given-informal are the only names skipped on purpose (KNOWN_FIELDS, IGNORED_FIELDS). - test_the_fields_partition_their_rendering: every word of a non-CLDR rendering is stated by exactly one field, so a left-out part fails. GND, Wikidata and conventions already held it. - Each conventions row carries fields or an unstated reason, and the builder refuses neither or both, so a row's fields cannot drop unseen behind the subset check. - _PARSERS gains shape 5, so a FAMILY_FIRST_GIVEN_LAST row no longer crashes the scorer. - Suzuki-san is stated after all: unspaced text matches by substring, so surname Suzuki + honorific -san partition it. My earlier reason for leaving it out was wrong. 26 of 31 rows now disagree. - Stale docstrings (two sources named of four), the ur note (a word of address, as its fields say), and the record of where today's reading lives. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Derek's call: the research groups the three as declining Kenyan parent-name markers whose last word may now be a family name, and its wa example names a mother, so wa is no surer a patronymic than the others. Its row stays in the corpus; 30 rows are stated, 25 disagree. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #686 / #691: the conventions corpus gets a fields file after all, so a change that moves one of its names shows whether the reading moved toward the expected one or away from it.
What.
conventions.py's 35 rows carryfieldsin CLDR's vocabulary, beside theirexpectnote.unstatedreason. The builder refuses a row with neither or both, so a row can't lose its fields unseen.build_conventions_corpus.pywritesconventions_fields.jsonl.cldr_agreement.pyscores it as a fourth source, recorded inconventions_agreement.json.honorific, scored insidesuffix.Where the readings come from. The research the rows were written from: the 2006 UK naming-practices guide and the Cultural Atlas report, both in the gitignored
docs/superpowers/, read back from the sessions that wrote #686. The fields follow the Wikidata source's conventions: apatronymicis written with its connective (bin Yusof) and scored insidegiven_names, and asurname2insidefamily.Your calls.
Bey,ssi,san,-san,úr) is a suffix. That matches the design notes and nameparser's reading ofさん; CLDR'sさん-as-title disagreement stays recorded.Kauris given2.Péternéis the given name.H. Budi Santosois title + one personal name.Ma.'sCruzis a surname2.Vda. de Morasits inside the surname.wais left unstated besidearap/ole.Left unstated (still in the corpus, still watched by the snapshot), since the scorer reads an unstated part as an empty role:
Abu KarimSzabóné Kiss Évaarap/ole/wa, whose last word is a parent's name or a family name depending on the familyGuards added in review, since this source is hand-written:
localeandgiven-informalare skipped on purpose. A mistypedhonorfichad been passing as agreement.test_the_fields_partition_their_rendering: every word of a non-CLDR rendering is stated by exactly one field. GND, Wikidata and conventions already held it.FAMILY_FIRST_GIVEN_LASTparser, so such a row no longer crashes it.Measured.
Harpreet Kaur Dhillon,Szabó Péterné,Kim Min Ho,Ivan Petrov Dimitrov,Lim Mei Ling Jessica.Hassan bin Yusof→ familybin Yusof), which makes the record a worklist for the 3.0 patronymic field. Titles read as given names (U Kyaw Win) and honorifics read as name words are the next groups.🤖 Generated with Claude Code