gh-124951: Optimize base64 encode & decode for an easy 2-3x speedup [no SIMD] #143262
Conversation
Add Tools/binasciibench/binasciibench.py benchmark for measuring base64
encoding/decoding throughput.
Optimize base64 encoding/decoding by eliminating loop-carried dependencies.
Key changes:
- Add base64_encode_trio() and base64_decode_quad() helper functions
that process complete groups independently
- Add base64_encode_fast() and base64_decode_fast() wrappers
- Update b2a_base64 and a2b_base64 to use fast path for complete groups
Performance gains (encode/decode speedup vs main, PGO builds):
64 bytes 64K 1M
Zen2: 1.1x/1.6x 1.6x/2.4x 1.4x/2.4x
Zen4: 1.2x/1.7x 1.6x/3.0x 1.5x/3.0x
M4: 1.3x/1.9x 2.3x/2.8x 2.4x/2.9x
RPi5-32: 1.4x/1.4x 2.4x/2.0x 2.0x/1.9x
Additional SIMD implementations (NEON, AVX-512 VBMI) can achieve
+50% to +1500% further gains and are planned for follow-on work.
Co-authored-by: Claude Opus 4.5 <[email protected]>
gpshead
force-pushed
the
opt-base64-cpu
branch
from
f15ad3e to
2d2be30
Compare
gpshead
changed the title
gpshead
changed the title
gpshead
changed the title
MSVC doesn't support forward declarations of arrays without explicit size. Move the table definition before the inline functions that use it, eliminating the need for a forward declaration. Co-authored-by: Claude Opus 4.5 <[email protected]>
Co-authored-by: Bénédikt Tran <[email protected]>
Add Py_ALIGNED(64) to both lookup tables to ensure each fits within a single L1 cache line, reducing potential cache misses during encoding/decoding loops. Co-authored-by: Claude Opus 4.5 <[email protected]>
Replace hardcoded '=' characters with the BASE64_PAD macro for consistency with the rest of the codebase. Co-authored-by: Claude Opus 4.5 <[email protected]>
Co-authored-by: Claude Opus 4.5 <[email protected]>
gpshead
force-pushed
the
opt-base64-cpu
branch
from
aad7a15 to
1f8ff74
Compare
serhiy-storchaka
left a comment
serhiy-storchaka
left a comment
There was a problem hiding this comment.
Looks pretty simple with large benefit.
BTW, I'm going to add support for ignorechars in the decoder, so it could support a multiline input without ignoring all other errors. The decoder will return on the fast path for each line.
| ----------------- | ||
|
|
||
| * CPython's underlying base64 implementation now encodes 2x faster and decodes 3x | ||
| faster thanks to simple CPU pipelining optimizations. |
|
|
||
|
|
||
| static const unsigned char table_a2b_base64[] = { | ||
| /* Align to 64 bytes to ensure table fits in a single L1 cache line */ |
There was a problem hiding this comment.
The entire 256-bytes table will not fit in a single L1 cache line.
It may be worth to align anyway, but the comment is incorrect.
| Py_ssize_t i; | ||
|
|
||
| for (i = 0; i < n_trios; i++) { | ||
| base64_encode_trio(in + i * 3, out + i * 4, table); |
There was a problem hiding this comment.
Is it faster than incrementing in and out by 3 and 4 correspondingly?
| if (ascii_len >= 4) { | ||
| Py_ssize_t fast_chars = base64_decode_fast(ascii_data, (Py_ssize_t)ascii_len, | ||
| bin_data, table_a2b_base64); | ||
| if (fast_chars > 0) { |
There was a problem hiding this comment.
Is this condition needed?
| Py_ssize_t fast_bytes = base64_encode_fast(bin_data, bin_len, ascii_data, | ||
| table_b2a_base64); | ||
| bin_data += fast_bytes; | ||
| ascii_data += (fast_bytes / 3) * 4; |
There was a problem hiding this comment.
I wonder, would not it be more efficient to return the number of groups, so you can avoid division. Although it can be below a noise.
| /* Check for padding - exit fast path to handle it properly. | ||
| * Four independent comparisons lets the compiler choose the optimal | ||
| * approach; on modern pipelined CPUs this is faster than bitmask tricks | ||
| * like XOR+SUB+AND for zero-detection which have data dependencies. | ||
| */ | ||
| if (inp[0] == BASE64_PAD || inp[1] == BASE64_PAD || | ||
| inp[2] == BASE64_PAD || inp[3] == BASE64_PAD) { | ||
| break; | ||
| } |
There was a problem hiding this comment.
For each group we have two checks. One here, with comparing all bytes to BASE64_PAD, and other in base64_decode_quad(), (v0 | v1 | v2 | v3) & 0xc0. Even if the former is much faster than the latter, aren't two checks slower then just one check? For most groups they are false, we can only have a benefit for the last group. For large data the benefit is much smaller than the cost which is proportional to the size of the data.
3 participants
Optimize base64 encoding/decoding by eliminating loop-carried dependencies. Key changes:
base64_encode_trio()andbase64_decode_quad()helper functions that process complete groups independentlybase64_encode_fast()andbase64_decode_fast()wrappersb2a_base64anda2b_base64to use fast path for complete groupsThe binasciibench I used measuring base64 encoding/decoding throughput is included in commit history, but i pulled it out of the PR in favor of adding to pyperformance.
Performance gains (encode/decode speedup vs main, PGO builds):
Additional SIMD implementations (NEON, AVX-512 VBMI) can achieve +50% (M4) to +1500% (!! Zen4) further gains and are planned for follow-on work if deemed simple to maintain.
Widely used third party libraries contain industry canonical SIMD accelerated variants such as simdutf (C++ based unfortunately) so the decision of how to link and use those and when is best kept separate.
This PR's simple pure better use of modern CPU functional unit pipelining wins make sense regardless.
Based on my exploratory work done in main...gpshead:cpython:claude/vectorize-base64-c-S7Hku
base64module: Link against SIMD library for 10x performance. #124951