HTTPError fp.read returns string instead of bytes

[gaborbernat]

Due to https://github.com/python/cpython/pull/99966/files

However, this returns a bytes stream on the live run:

from http import HTTPStatus
from mailbox import Message
from urllib.error import HTTPError
from urllib.request import urlopen

try:
    urlopen("http://asadsad.sd")
except HTTPError as exception:
    content = exception.fp.read()
    print(type(content))

error = HTTPError(url="url", code=HTTPStatus.IM_A_TEAPOT, msg="msg", hdrs=Message(), fp=None)
print(type(error.fp.read()))

<class 'bytes'>
<class 'str'>

Linked PRs

• gh-101936: Update the default value of fp from io.StringIO to io.BytesIO #102100
• [3.11] gh-101936: Update the default value of fp from io.StringIO to io.BytesIO (gh-102100) #102117
• [3.10] gh-101936: Update the default value of fp from io.StringIO to io.BytesIO (gh-102100) #102118

[sobolevn]

cc @corona10

[corona10]

@gaborbernat @sobolevn
Thanks, I will take a look and use this issue through the internal OSS sprint as the good first bug.
I am mentoring them this year Thank you for your understanding!

[longvhdeveloper]

I'll take it

[longvhdeveloper]

@corona10 I ran code in live run the code

from mailbox import Message
from urllib.error import HTTPError
from urllib.request import urlopen

try:
    urlopen("http://asadsad.sd")
except HTTPError as exception:
    content = exception.fp.read()
    print(type(content))

error = HTTPError(url="url", code=HTTPStatus.IM_A_TEAPOT, msg="msg", hdrs=Message(), fp=None)
print(type(error.fp.read()))

I checked type of fp

print(exception.fp)

is <class 'http.client.HTTPResponse'>
it is not StringIO, that mean fp is not None. And if you call read() method it will return bytes

cpython/Lib/urllib/request.py

Line 639 in dc8a868

raise HTTPError(req.full_url, code, msg, hdrs, fp)

https://docs.python.org/3/library/http.client.html#http.client.HTTPResponse.read

Do I change StringIO to BytesIO when fp is None ?

[gaborbernat]

The title is incorrect, should be read returns string instead of bytes when fp is None.

[corona10]

@longvhdeveloper

This script might explain better.

from urllib.request import urlopen
from urllib.error import HTTPError

try:
    urlopen("https://httpstat.us/500")
except HTTPError as exception:
    content = exception.fp.read()
    print(type(exception.fp))
    print(type(content))

Do I change StringIO to BytesIO when fp is None ?

Yes please go ahead with nice test code.

[corona10]

The title is incorrect, should be read returns string instead of bytes when fp is None.

Thanks, I modified the title.