asyncio: start_tls() loses buffered data from StreamReader

[kasimov-maxim]

Bug report

Bug description:

Bug description

When using StreamWriter.start_tls() (or loop.start_tls()) to upgrade a connection to TLS mid-stream, any data already buffered in the StreamReader is lost.

This commonly occurs when implementing servers that support the PROXY protocol (e.g., behind HAProxy with send-proxy). In this scenario:

1. The proxy sends the PROXY protocol header followed immediately by the TLS ClientHello in the same TCP segment
2. The server reads the PROXY header using reader.readline()
3. The TLS ClientHello data is now buffered in StreamReader._buffer
4. When writer.start_tls() is called, a new SSLProtocol is created, but the buffered data is not transferred to it
5. The TLS handshake hangs waiting for ClientHello data that was already received but lost

Reproducible example

Server (server.py)

import asyncio
import ssl

async def handle_client(reader, writer):
    # Read PROXY protocol header
    proxy_header = await reader.readline()
    print(f"PROXY header: {proxy_header}")

    # Upgrade to TLS - this is where data loss occurs
    context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
    context.load_cert_chain('server.crt', 'server.key')
    
    await writer.start_tls(context)
    print("TLS handshake completed")  # Never reached - hangs here

    data = await reader.read(1024)
    print(f"Received: {data}")
    
    writer.close()
    await writer.wait_closed()

async def main():
    server = await asyncio.start_server(handle_client, '127.0.0.1', 8000)
    async with server:
        await server.serve_forever()

asyncio.run(main())

HAProxy config (haproxy.cfg)

frontend proxy_frontend
    mode tcp
    bind *:5002
    default_backend proxy_backend

backend proxy_backend
    mode tcp
    server app 127.0.0.1:8000 send-proxy

Client command

echo "Test message" | openssl s_client -connect 127.0.0.1:5002

Expected behavior

TLS handshake completes and server receives "Test message".

Actual behavior

TLS handshake hangs indefinitely because the TLS ClientHello data buffered in StreamReader is lost when start_tls() is called.

Additional context

I have a fix ready with tests and will submit a PR shortly.

The fix transfers buffered data from StreamReader._buffer to SSLProtocol._incoming before the TLS handshake begins.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-142352: Fix start_tls() to transfer buffered data from StreamReader #142354

[fizzi01]

Your solution works perfectly!

[kumaraditya303]

Does this happens with uvloop as well? ideally asyncio should handle this similar to how it is done in uvloop.

[fizzi01]

Does this happens with uvloop as well? ideally asyncio should handle this similar to how it is done in uvloop.

Yes it does