<feed xmlns='http://www.w3.org/2005/Atom'>
<title>mirrors/foot.git/vt.c, branch 1.8.0</title>
<subtitle>A fast, lightweight and minimalistic Wayland terminal emulator</subtitle>
<id>https://git.0x0f.dev/mirrors/foot.git/atom?h=1.8.0</id>
<link rel='self' href='https://git.0x0f.dev/mirrors/foot.git/atom?h=1.8.0'/>
<link rel='alternate' type='text/html' href='https://git.0x0f.dev/mirrors/foot.git/'/>
<updated>2021-06-24T17:36:39Z</updated>
<entry>
<title>vt: utf8: rotate instead of just shifting when updating compose key</title>
<updated>2021-06-24T17:36:39Z</updated>
<author>
<name>Daniel Eklöf</name>
<email>daniel@ekloef.se</email>
</author>
<published>2021-06-24T17:36:39Z</published>
<link rel='alternate' type='text/html' href='https://git.0x0f.dev/mirrors/foot.git/commit/?id=3bad062f8a8f73b9271d42cc00b7c8a74fa82617'/>
<id>urn:sha1:3bad062f8a8f73b9271d42cc00b7c8a74fa82617</id>
<content type='text'>
This reduces the number of collisions in even more workloads.
</content>
</entry>
<entry>
<title>vt: improved key hash algorithm -&gt; reduces number of key collisions</title>
<updated>2021-06-24T17:18:06Z</updated>
<author>
<name>Daniel Eklöf</name>
<email>daniel@ekloef.se</email>
</author>
<published>2021-06-24T17:18:06Z</published>
<link rel='alternate' type='text/html' href='https://git.0x0f.dev/mirrors/foot.git/commit/?id=88ce0e437518eee07de99ac51474e03e897bf287'/>
<id>urn:sha1:88ce0e437518eee07de99ac51474e03e897bf287</id>
<content type='text'>
</content>
</entry>
<entry>
<title>composed: insert: require key to be unique</title>
<updated>2021-06-24T17:12:25Z</updated>
<author>
<name>Daniel Eklöf</name>
<email>daniel@ekloef.se</email>
</author>
<published>2021-06-24T17:12:25Z</published>
<link rel='alternate' type='text/html' href='https://git.0x0f.dev/mirrors/foot.git/commit/?id=f20956ff1bdfb266ae9c690bd2961acf7b3f4845'/>
<id>urn:sha1:f20956ff1bdfb266ae9c690bd2961acf7b3f4845</id>
<content type='text'>
</content>
</entry>
<entry>
<title>vt: codespell: bumb -&gt; bump</title>
<updated>2021-06-24T15:30:50Z</updated>
<author>
<name>Daniel Eklöf</name>
<email>daniel@ekloef.se</email>
</author>
<published>2021-06-24T11:53:19Z</published>
<link rel='alternate' type='text/html' href='https://git.0x0f.dev/mirrors/foot.git/commit/?id=415ecfc6fa0d0d7c2d4f8c455b770b781f2a5ef3'/>
<id>urn:sha1:415ecfc6fa0d0d7c2d4f8c455b770b781f2a5ef3</id>
<content type='text'>
</content>
</entry>
<entry>
<title>composed: store compose chains in a binary search tree</title>
<updated>2021-06-24T15:30:49Z</updated>
<author>
<name>Daniel Eklöf</name>
<email>daniel@ekloef.se</email>
</author>
<published>2021-06-24T11:17:07Z</published>
<link rel='alternate' type='text/html' href='https://git.0x0f.dev/mirrors/foot.git/commit/?id=fe8ca23cfee77a6cac433892755dd7455e354a9d'/>
<id>urn:sha1:fe8ca23cfee77a6cac433892755dd7455e354a9d</id>
<content type='text'>
The previous implementation stored compose chains in a dynamically
allocated array. Adding a chain was easy: resize the array and append
the new chain at the end. Looking up a compose chain given a compose
chain key/index was also easy: just index into the array.

However, searching for a pre-existing chain given a codepoint sequence
was very slow. Since the array wasn’t sorted, we typically had to scan
through the entire array, just to realize that there is no
pre-existing chain, and that we need to add a new one.

Since this happens for *each* codepoint in a grapheme cluster, things
quickly became really slow.

Things were ok:ish as long as the compose chain struct was small, as
that made it possible to hold all the chains in the cache. Once the
number of chains reached a certain point, or when we were forced to
bump maximum number of allowed codepoints in a chain, we started
thrashing the cache and things got much much worse.

So what can we do?

We can’t sort the array, because

a) that would invalidate all existing chain keys in the grid (and
iterating the entire scrollback and updating compose keys is *not* an
option).

b) inserting a chain becomes slow as we need to first find _where_ to
insert it, and then memmove() the rest of the array.

This patch uses a binary search tree to store the chains instead of a
simple array.

The tree is sorted on a “key”, which is the XOR of all codepoints,
truncated to the CELL_COMB_CHARS_HI-CELL_COMB_CHARS_LO range.

The grid now stores CELL_COMB_CHARS_LO+key, instead of
CELL_COMB_CHARS_LO+index.

Since the key is truncated, collisions may occur. This is handled by
incrementing the key by 1.

Lookup is of course slower than before, O(log n) instead of
O(1).

Insertion is slightly slower as well: technically it’s O(log n)
instead of O(1). However, we also need to take into account the
re-allocating the array will occasionally force a full copy of the
array when it cannot simply be growed.

But finding a pre-existing chain is now *much* faster: O(log n)
instead of O(n). In most cases, the first lookup will either
succeed (return a true match), or fail (return NULL). However, since
key collisions are possible, it may also return false matches. This
means we need to verify the contents of the chain before deciding to
use it instead of inserting a new chain. But remember that this
comparison was being done for each and every chain in the previous
implementation.

With lookups being much faster, and in particular, no longer requiring
us to check the chain contents for every singlec chain, we can now use
a dynamically allocated ‘chars’ array in the chain. This was
previously a hardcoded array of 10 chars.

Using a dynamic allocated array means looking in the array is slower,
since we now need two loads: one to load the pointer, and a second to
load _from_ the pointer.

As a result, the base size of a compose chain (i.e. an “empty” chain)
has now been reduced from 48 bytes to 32. A chain with two codepoints
is 40 bytes. This means we have up to 4 codepoints while still using
less, or the same amount, of memory as before.

Furthermore, the Unicode random test (i.e. write random “unicode”
chars) is now **faster** than current master (i.e. before text-shaping
support was added), **with** test-shaping enabled. With text-shaping
disabled, we’re _even_ faster.
</content>
</entry>
<entry>
<title>vt: utf8: don’t scan *all* previous chains</title>
<updated>2021-06-24T15:30:48Z</updated>
<author>
<name>Daniel Eklöf</name>
<email>daniel@ekloef.se</email>
</author>
<published>2021-06-18T15:53:15Z</published>
<link rel='alternate' type='text/html' href='https://git.0x0f.dev/mirrors/foot.git/commit/?id=81131e3a8757658be264e2069c75406993b8e876'/>
<id>urn:sha1:81131e3a8757658be264e2069c75406993b8e876</id>
<content type='text'>
When checking if we already have a compose chain for the current
sequence of characters, don’t search the list from the beginning,
unless we have to.

Taking the following things into consideration:

* New compose chains are always appended at the end of the list
* If the current sequence is 3 or more characters, it *must* consist
  of an existing compose chain, plus the new character.

Thus, when searching, start at index 0 if we only have two characters,
since then the base cell originally contained a regular base
character, and not a compose chain. I.e. the new chain may be
_anywhere_ in the chain list.

If however we have a sequence of three or more characters, start at
the index the *base* chain was at. If the chain we’re searching for
exists, it *must* have been added *after* the base chain, and thus
it *must* be located *after* the base chain in the chain list.
</content>
</entry>
<entry>
<title>vt: utf8: de-duplicate; jump to end of function to print to grid</title>
<updated>2021-06-24T15:30:48Z</updated>
<author>
<name>Daniel Eklöf</name>
<email>daniel@ekloef.se</email>
</author>
<published>2021-06-18T15:40:24Z</published>
<link rel='alternate' type='text/html' href='https://git.0x0f.dev/mirrors/foot.git/commit/?id=e81d1845bf765055672b76028525221edefd4aa6'/>
<id>urn:sha1:e81d1845bf765055672b76028525221edefd4aa6</id>
<content type='text'>
</content>
</entry>
<entry>
<title>vt: utf8-print: don’t build a compose chain on a zero-width base character</title>
<updated>2021-06-24T15:30:47Z</updated>
<author>
<name>Daniel Eklöf</name>
<email>daniel@ekloef.se</email>
</author>
<published>2021-06-15T07:00:18Z</published>
<link rel='alternate' type='text/html' href='https://git.0x0f.dev/mirrors/foot.git/commit/?id=dc5019a535f3d678332ca1d68865356c970a200c'/>
<id>urn:sha1:dc5019a535f3d678332ca1d68865356c970a200c</id>
<content type='text'>
</content>
</entry>
<entry>
<title>vt: utf8-print: check base character before count when looking for existing compose chain</title>
<updated>2021-06-24T15:30:47Z</updated>
<author>
<name>Daniel Eklöf</name>
<email>daniel@ekloef.se</email>
</author>
<published>2021-06-15T06:40:23Z</published>
<link rel='alternate' type='text/html' href='https://git.0x0f.dev/mirrors/foot.git/commit/?id=f865612667d090b584c1ea6cc20f32fcffd828d9'/>
<id>urn:sha1:f865612667d090b584c1ea6cc20f32fcffd828d9</id>
<content type='text'>
Count is more likely to be the same for many chains. Thus we’re likely
to fail sooner by checking the base character first.
</content>
</entry>
<entry>
<title>vt: don’t call wcwidth() on all combining characters every time we add</title>
<updated>2021-06-24T15:30:46Z</updated>
<author>
<name>Daniel Eklöf</name>
<email>daniel@ekloef.se</email>
</author>
<published>2021-06-15T05:38:27Z</published>
<link rel='alternate' type='text/html' href='https://git.0x0f.dev/mirrors/foot.git/commit/?id=57e636dd8e3be0e75492f226e036943989422b32'/>
<id>urn:sha1:57e636dd8e3be0e75492f226e036943989422b32</id>
<content type='text'>
We already have all the widths needed to calculate the new one; it’s
the base characters width (base_width), or the previous combining
chain’s width (composed-&gt;width) plus the new characters’s
width (width).
</content>
</entry>
<entry>
<title>vt: presentation selectors may be anywhere in the cluster</title>
<updated>2021-06-24T15:30:46Z</updated>
<author>
<name>Daniel Eklöf</name>
<email>daniel@ekloef.se</email>
</author>
<published>2021-06-15T05:25:38Z</published>
<link rel='alternate' type='text/html' href='https://git.0x0f.dev/mirrors/foot.git/commit/?id=09431dd15c0e5d67adde766deb5f685a2918bb18'/>
<id>urn:sha1:09431dd15c0e5d67adde766deb5f685a2918bb18</id>
<content type='text'>
</content>
</entry>
<entry>
<title>vt: don’t force cols=2 when we see an emoji variant selector</title>
<updated>2021-06-24T15:30:45Z</updated>
<author>
<name>Daniel Eklöf</name>
<email>daniel@ekloef.se</email>
</author>
<published>2021-05-30T17:36:59Z</published>
<link rel='alternate' type='text/html' href='https://git.0x0f.dev/mirrors/foot.git/commit/?id=6c70cd9366db76f10d822252151a782f638d1f45'/>
<id>urn:sha1:6c70cd9366db76f10d822252151a782f638d1f45</id>
<content type='text'>
Fish appears to be the only shell expecting this. The rest probably
just does wcswidth(), like usual.
</content>
</entry>
<entry>
<title>vt: cache grapheme cluster width in composed struct</title>
<updated>2021-06-24T15:30:45Z</updated>
<author>
<name>Daniel Eklöf</name>
<email>daniel@ekloef.se</email>
</author>
<published>2021-05-27T18:07:28Z</published>
<link rel='alternate' type='text/html' href='https://git.0x0f.dev/mirrors/foot.git/commit/?id=0a9531ac6caff2ad6abfbb804ec27cade4a92e7a'/>
<id>urn:sha1:0a9531ac6caff2ad6abfbb804ec27cade4a92e7a</id>
<content type='text'>
* Use regular wcswidth() to calculate the width
* Explicitly set to ‘2’ if we see a emoji variant selector
* Cache the result in the composed struct
</content>
</entry>
<entry>
<title>wip: grapheme shaping</title>
<updated>2021-06-24T15:30:45Z</updated>
<author>
<name>Daniel Eklöf</name>
<email>daniel@ekloef.se</email>
</author>
<published>2020-08-20T17:25:35Z</published>
<link rel='alternate' type='text/html' href='https://git.0x0f.dev/mirrors/foot.git/commit/?id=b9ef703eb14ba2357b847b62a715b4e97f690209'/>
<id>urn:sha1:b9ef703eb14ba2357b847b62a715b4e97f690209</id>
<content type='text'>
</content>
</entry>
<entry>
<title>Merge branch 'charset-shift-fixes'</title>
<updated>2021-06-09T09:18:52Z</updated>
<author>
<name>Craig Barnes</name>
<email>craigbarnes@protonmail.com</email>
</author>
<published>2021-06-09T09:18:52Z</published>
<link rel='alternate' type='text/html' href='https://git.0x0f.dev/mirrors/foot.git/commit/?id=2a75da41436be42a1b1d0ce32994ef26480139ae'/>
<id>urn:sha1:2a75da41436be42a1b1d0ce32994ef26480139ae</id>
<content type='text'>
</content>
</entry>
<entry>
<title>terminal: add 'charset_designator' enum to make code more self-documenting</title>
<updated>2021-06-09T09:00:25Z</updated>
<author>
<name>Craig Barnes</name>
<email>craigbarnes@protonmail.com</email>
</author>
<published>2021-06-09T08:51:48Z</published>
<link rel='alternate' type='text/html' href='https://git.0x0f.dev/mirrors/foot.git/commit/?id=e030a2ca08d92eb431e9b8e494c63affeeb8c0e1'/>
<id>urn:sha1:e030a2ca08d92eb431e9b8e494c63affeeb8c0e1</id>
<content type='text'>
This commit also renames the term_set_single_shift_ascii_printer()
function to term_single_shift(), since the former is overly verbose
and not really even accurate.
</content>
</entry>
<entry>
<title>vt: fix SS2/SS3 escape sequences to act correctly as single shifts</title>
<updated>2021-06-08T20:09:40Z</updated>
<author>
<name>Craig Barnes</name>
<email>craigbarnes@protonmail.com</email>
</author>
<published>2021-06-08T20:09:40Z</published>
<link rel='alternate' type='text/html' href='https://git.0x0f.dev/mirrors/foot.git/commit/?id=a2c9c56f1962ce02e71582c06f389e53d1869e90'/>
<id>urn:sha1:a2c9c56f1962ce02e71582c06f389e53d1869e90</id>
<content type='text'>
These sequences are supposed to affect the next printable ASCII
character and then reset to the previous character set, but before
this commit they were behaving like locking shifts.
</content>
</entry>
<entry>
<title>vt: add support for LS2 and LS3 locking shifts</title>
<updated>2021-06-08T20:06:18Z</updated>
<author>
<name>Craig Barnes</name>
<email>craigbarnes@protonmail.com</email>
</author>
<published>2021-06-08T20:06:18Z</published>
<link rel='alternate' type='text/html' href='https://git.0x0f.dev/mirrors/foot.git/commit/?id=e72e8b1b8e7ffcd32c97260e6a4ef2466591f628'/>
<id>urn:sha1:e72e8b1b8e7ffcd32c97260e6a4ef2466591f628</id>
<content type='text'>
</content>
</entry>
<entry>
<title>vt: TAB: don’t print a ‘\t’ to the grid if the *current* cell isn’t empty</title>
<updated>2021-06-08T17:53:26Z</updated>
<author>
<name>Daniel Eklöf</name>
<email>daniel@ekloef.se</email>
</author>
<published>2021-06-07T19:16:38Z</published>
<link rel='alternate' type='text/html' href='https://git.0x0f.dev/mirrors/foot.git/commit/?id=9d3351472d9d6ef9a4314906fe71bf5fa3ed77bf'/>
<id>urn:sha1:9d3351472d9d6ef9a4314906fe71bf5fa3ed77bf</id>
<content type='text'>
If the cursor is already at the right edge, our logic that checked for
non-empty cells failed; it didn’t check the current cell.

Fix by initializing ‘emit_tab_char’ to true/false, depending on the
contents of the current cell.
</content>
</entry>
<entry>
<title>vt: emit a tab character if all cells between cursor and tab stop are empty</title>
<updated>2021-06-08T17:53:26Z</updated>
<author>
<name>Daniel Eklöf</name>
<email>daniel@ekloef.se</email>
</author>
<published>2021-06-05T20:48:20Z</published>
<link rel='alternate' type='text/html' href='https://git.0x0f.dev/mirrors/foot.git/commit/?id=94b549f93e8cc4372e5ab7dc9c832219f337819c'/>
<id>urn:sha1:94b549f93e8cc4372e5ab7dc9c832219f337819c</id>
<content type='text'>
TAB (\t) move the cursor to the next tab stop. That’s it, according to
the specification.

However, many terminal emulators try to keep tabs in the grid, to be
able to e.g. copy them. That is, copying a text chunk containing tabs
should result in tabs being pasted, not spaces.

In order to do that, we need to print a tab character to the grid. To
improve text reflow of tabs, we also print spaces to the subsequent
cells, up until (but not including) the next tab stop.

However, we can only do this if all the cells between the cursor and
the next tab stop are empty, since (obviously), we cannot overwrite
pre-existing characters.

Finally, while some fonts render tabs as spaces (i.e. an empty glyph),
some use a glyph representing “unprintable” characters, or
similar. Thus, we need to exclude cells with tab characters when
rendering.
</content>
</entry>
</feed>
