Skip to content

Commit 24b499d

Browse files
candpixiecandpixie
committed
Add scripts/scaffold_krn.py so contributing a song is only the listening
Hand-writing Humdrum was the boring half of adding a song: three tab separated spines, one character per row, jyutping on every one. This takes plain lyrics, one phrase per line with # section marks, and emits the whole structure with the jyutping looked up. What is left is replacing each placeholder pitch with the note actually sung. It reports every character it could not read, marked TODO, and every polyphone it had to guess, so it is clear which readings need checking rather than leaving wrong jyutping to be discovered later in the model. check_krn.py now fails on any file still carrying the SKELETON marker, so an untranscribed draft cannot reach the corpus by accident. Verified both ways: exit 1 while the marker is present, exit 0 once the pitches are real and it is removed. Documented in corpus/README.md and CONTRIBUTING.md.
1 parent 7cea262 commit 24b499d

3 files changed

Lines changed: 142 additions & 1 deletion

File tree

‎CONTRIBUTING.md‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,10 +12,16 @@ it. Anything outside that is new information. See
1212
[`corpus/TEMPLATE.krn`](corpus/TEMPLATE.krn) to start from.
1313

1414
```bash
15-
cp corpus/TEMPLATE.krn corpus/X0001.krn # edit it
15+
python scripts/scaffold_krn.py lyrics.txt --id X0001 --title 歌名 > corpus/X0001.krn
16+
# replace each placeholder pitch with the note actually sung, drop the
17+
# SKELETON line, then:
1618
python scripts/check_krn.py corpus/
1719
```
1820

21+
`scaffold_krn.py` writes the Humdrum structure and looks up the jyutping, so
22+
the only work left is the listening. It flags unreadable characters and every
23+
polyphone it guessed.
24+
1925
Open a pull request with the `.krn` file. One song is a real contribution.
2026

2127
Transcriptions are released CC BY 4.0 so they pool with the upstream corpus.

‎scripts/check_krn.py‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@
2222
SECTIONS = {"verse", "prechorus", "chorus", "bridge", "coda", "interlude",
2323
"outro", "intro", "strophe", "guitar solo", "refrain"}
2424
REQUIRED_RECORDS = ["OTL", "OTA"]
25+
SKELETON_MARKER = "SKELETON"
2526

2627

2728
def check_file(path):
@@ -41,6 +42,11 @@ def check_file(path):
4142
if line.startswith("!!!"):
4243
key, _, value = line[3:].partition(":")
4344
records[key.strip()] = value.strip()
45+
if SKELETON_MARKER in value:
46+
errors.append(
47+
f"{os.path.basename(path)}: still a scaffold_krn.py "
48+
f"skeleton. Replace the placeholder pitches with the notes "
49+
f"actually sung, then delete the !!!ONB: SKELETON line.")
4450
continue
4551
if line.startswith("!"):
4652
continue

‎scripts/scaffold_krn.py‎

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
"""Turn a page of lyrics into a .krn skeleton, so only the pitches are left.
2+
3+
Hand-writing Humdrum is the boring half of contributing a song: three tab
4+
separated spines, one character per row, jyutping on every one. This does that
5+
part and looks up the jyutping, leaving you to replace each placeholder pitch
6+
with the note actually sung.
7+
8+
Input is plain text, one phrase per line. A line beginning with # marks a
9+
section:
10+
11+
# verse
12+
今日天氣真係好
13+
我哋一齊去食飯
14+
# chorus
15+
佢話唔記得帶錢
16+
17+
The output carries a SKELETON marker, and scripts/check_krn.py refuses any file
18+
that still has it, so an untranscribed draft cannot reach the corpus by
19+
accident. Delete that line once the pitches are real.
20+
21+
Usage:
22+
python scripts/scaffold_krn.py LYRICS.txt --id X0001 --title 歌名 \\
23+
--singer 歌手 --composer 作曲 --lyricist 作詞 --year 2026 > corpus/X0001.krn
24+
"""
25+
26+
import argparse
27+
import os
28+
import sys
29+
30+
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
31+
32+
from cantojam.jyutping import Lexicon, is_han # noqa: E402
33+
34+
SKELETON = "!!!ONB: SKELETON, pitches not yet transcribed"
35+
PLACEHOLDER = "4c"
36+
SECTIONS = {"verse", "prechorus", "chorus", "bridge", "coda", "intro",
37+
"outro", "interlude", "refrain"}
38+
39+
40+
def main():
41+
parser = argparse.ArgumentParser(
42+
description=__doc__,
43+
formatter_class=argparse.RawDescriptionHelpFormatter)
44+
parser.add_argument("lyrics", help="plain text, one phrase per line")
45+
parser.add_argument("--id", required=True, help="song ID, e.g. X0001")
46+
parser.add_argument("--title", required=True)
47+
parser.add_argument("--singer", default="")
48+
parser.add_argument("--composer", default="")
49+
parser.add_argument("--lyricist", default="")
50+
parser.add_argument("--arranger", default="")
51+
parser.add_argument("--year", default="")
52+
parser.add_argument("--key", default="*k[b-]\t*\t*",
53+
help="Humdrum key signature line")
54+
parser.add_argument("--tonic", default="*F:")
55+
parser.add_argument("--meter", default="*M4/4")
56+
parser.add_argument("--tempo", default="*MM72")
57+
parser.add_argument("--override", action="append", metavar="字=jyutping",
58+
help="pin a reading for this song")
59+
args = parser.parse_args()
60+
61+
overrides = {}
62+
for pair in args.override or []:
63+
char, _, reading = pair.partition("=")
64+
if not reading:
65+
sys.exit(f"bad override {pair!r}, expected 字=jyutping")
66+
overrides[char.strip()] = reading.strip()
67+
68+
lexicon = Lexicon(overrides)
69+
with open(args.lyrics, encoding="utf-8") as handle:
70+
lines = [line.rstrip("\n") for line in handle]
71+
72+
out = [SKELETON, f"!!!OTL: {args.title}", f"!!!OTA: {args.id}"]
73+
for tag, value in (("RRD", args.year), ("MGN", args.singer),
74+
("COM", args.composer), ("LYR", args.lyricist),
75+
("LAR", args.arranger)):
76+
if value:
77+
out.append(f"!!!{tag}: {value}")
78+
79+
out += ["**kern\t**text\t**jyutping", "*clefGv2\t*\t*", args.key,
80+
f"{args.tonic}\t*\t*", f"{args.meter}\t*\t*",
81+
f"{args.tempo}\t*\t*"]
82+
83+
unknown, ambiguous, bar, syllables = [], [], 0, 0
84+
for line in lines:
85+
stripped = line.strip()
86+
if not stripped:
87+
continue
88+
if stripped.startswith("#"):
89+
name = stripped.lstrip("#").strip()
90+
if name and name not in SECTIONS:
91+
print(f"warning: unrecognised section {name!r}",
92+
file=sys.stderr)
93+
out.append(f"*>{name}\t*>{name}\t*>{name}")
94+
continue
95+
96+
out.append(f"!! {stripped}") # the phrase, for your eyes only
97+
for char in stripped:
98+
if not is_han(char):
99+
continue
100+
reading = lexicon.lookup(char)
101+
if reading is None:
102+
unknown.append(char)
103+
reading = "TODO"
104+
elif lexicon.is_ambiguous(char):
105+
ambiguous.append(f"{char}({'/'.join(lexicon.readings(char))})")
106+
out.append(f"{PLACEHOLDER}\t{char}\t{reading}")
107+
syllables += 1
108+
bar += 1
109+
out.append(f"={bar}\t={bar}\t={bar}")
110+
111+
out.append("*-\t*-\t*-")
112+
print("\n".join(out))
113+
114+
print(f"\n{syllables} syllables over {bar} phrases", file=sys.stderr)
115+
if unknown:
116+
print(f"jyutping missing for {len(set(unknown))} character(s), marked "
117+
f"TODO: {''.join(sorted(set(unknown)))}", file=sys.stderr)
118+
print(" add them to cantojam/data/colloquial.json, or fill by hand",
119+
file=sys.stderr)
120+
if ambiguous:
121+
shown = ", ".join(sorted(set(ambiguous))[:8])
122+
print(f"{len(set(ambiguous))} polyphone(s), defaulted to the corpus's "
123+
f"most frequent reading. Check these: {shown}", file=sys.stderr)
124+
print(f"\nnow replace every {PLACEHOLDER} with the pitch actually sung, "
125+
f"then delete the SKELETON line.", file=sys.stderr)
126+
127+
128+
if __name__ == "__main__":
129+
main()

0 commit comments

Comments
 (0)