aboutsummaryrefslogtreecommitdiff
path: root/scripts
diff options
context:
space:
mode:
authorDaniel Eklöf <daniel@ekloef.se>2020-05-02 15:29:00 +0000
committerDaniel Eklöf <daniel@ekloef.se>2020-05-02 15:29:00 +0000
commitd945b68b734427a03d114716ff8aeac7998a7c57 (patch)
tree93c73c5ab615ccf66244713d15a2a7a2fd139e0d /scripts
parentunicode-combining: don't limit ourselves to the (western) diacritics blocks (diff)
unicode-combine: remove utf8proc dependency
We only used utf8proc to try to pre-compose a glyph from a base and combining character. We can do this ourselves by using a pre-compiled table of valid pre-compositions. This table isn't _that_ big, and binary searching it is fast. That is, for a very small amount of code, and not too much extra RO data, we can get rid of the utf8proc dependency.
Diffstat (limited to 'scripts')
-rwxr-xr-xscripts/generate-unicode-precompose.sh8
1 files changed, 8 insertions, 0 deletions
diff --git a/scripts/generate-unicode-precompose.sh b/scripts/generate-unicode-precompose.sh
new file mode 100755
index 00000000..830e9379
--- /dev/null
+++ b/scripts/generate-unicode-precompose.sh
@@ -0,0 +1,8 @@
+#!/usr/bin/sh
+
+cut - -d ";" -f 1,6 |
+ grep ";[0-9,A-F]" | grep " " |
+ sed -e "s/ /, 0x/;s/^/{ 0x/;s/;/, 0x/;s/$/},/" |
+ sed -e "s,0x\(....\)\([^0-9A-Fa-f]\),0x0\1\2,g" |
+ (sort -k 3 || sort +2) |
+ sed -e "s,0x0\(...[0-9A-Fa-f]\),0x\1,g"