diff options
| author | Daniel Eklöf <daniel@ekloef.se> | 2020-05-02 15:29:00 +0000 |
|---|---|---|
| committer | Daniel Eklöf <daniel@ekloef.se> | 2020-05-02 15:29:00 +0000 |
| commit | d945b68b734427a03d114716ff8aeac7998a7c57 (patch) | |
| tree | 93c73c5ab615ccf66244713d15a2a7a2fd139e0d /scripts | |
| parent | unicode-combining: don't limit ourselves to the (western) diacritics blocks (diff) | |
unicode-combine: remove utf8proc dependency
We only used utf8proc to try to pre-compose a glyph from a base and
combining character.
We can do this ourselves by using a pre-compiled table of valid
pre-compositions. This table isn't _that_ big, and binary searching it
is fast.
That is, for a very small amount of code, and not too much extra RO
data, we can get rid of the utf8proc dependency.
Diffstat (limited to 'scripts')
| -rwxr-xr-x | scripts/generate-unicode-precompose.sh | 8 |
1 files changed, 8 insertions, 0 deletions
diff --git a/scripts/generate-unicode-precompose.sh b/scripts/generate-unicode-precompose.sh new file mode 100755 index 00000000..830e9379 --- /dev/null +++ b/scripts/generate-unicode-precompose.sh @@ -0,0 +1,8 @@ +#!/usr/bin/sh + +cut - -d ";" -f 1,6 | + grep ";[0-9,A-F]" | grep " " | + sed -e "s/ /, 0x/;s/^/{ 0x/;s/;/, 0x/;s/$/},/" | + sed -e "s,0x\(....\)\([^0-9A-Fa-f]\),0x0\1\2,g" | + (sort -k 3 || sort +2) | + sed -e "s,0x0\(...[0-9A-Fa-f]\),0x\1,g" |
