Unicode Marker in Python 2

[bpmweel]

Description

I'm trying to use a unicode string as a marker in both python 2 and 3. In python 3 this works fine, but in python 2 I get the following error (Full trace at the end):

UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-24: ordinal not in range(128)

Using encode('utf-8') on this marker results in the error: AssertionError: Markers are required to be 1 char

The trace shows that the string, which is already unicode, is being encoded into unicode again. Perhaps this is superfluous?

Should I call bar in a different way?

Code

import progressbar
from progressbar import Bar

widgets = [Bar(marker=u'\u2588')]
progress_bar = progressbar.ProgressBar(
                max_value=10, widgets=widgets)
progress_bar.start()
progress_bar.update(1)

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/bweel/anaconda2/envs/boatswain2/lib/python2.7/site-packages/progressbar/bar.py", line 550, in update
    StdRedirectMixin.update(self, value=value)
  File "/home/bweel/anaconda2/envs/boatswain2/lib/python2.7/site-packages/progressbar/bar.py", line 143, in update
    DefaultFdMixin.update(self, value=value)
  File "/home/bweel/anaconda2/envs/boatswain2/lib/python2.7/site-packages/progressbar/bar.py", line 48, in update
    self.fd.write('\r' + self._format_line())
  File "/home/bweel/anaconda2/envs/boatswain2/lib/python2.7/site-packages/progressbar/bar.py", line 481, in _format_line
    widgets = ''.join(self._format_widgets())
  File "/home/bweel/anaconda2/envs/boatswain2/lib/python2.7/site-packages/progressbar/bar.py", line 472, in _format_widgets
    widget_output = widget(self, data, portion)
  File "/home/bweel/anaconda2/envs/boatswain2/lib/python2.7/site-packages/progressbar/widgets.py", line 669, in __call__
    marker = converters.to_unicode(self.marker(progress, data, width))
  File "/home/bweel/anaconda2/envs/boatswain2/lib/python2.7/site-packages/python_utils/converters.py", line 178, in to_unicode
    input_ = input_.decode(encoding, errors)
  File "/home/bweel/anaconda2/envs/boatswain2/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-15: ordinal not in range(128)

import progressbar
from progressbar import Bar

widgets = [Bar(marker=u'\u2588'.encode('utf-8'))]

  File "<stdin>", line 1, in <module>
  File "/home/bweel/anaconda2/envs/boatswain2/lib/python2.7/site-packages/progressbar/widgets.py", line 655, in __init__
    self.marker = create_marker(marker)
  File "/home/bweel/anaconda2/envs/boatswain2/lib/python2.7/site-packages/progressbar/widgets.py", line 40, in create_marker
    assert len(marker) == 1, 'Markers are required to be 1 char'
AssertionError: Markers are required to be 1 char

Versions

• Python version: 2.7.12 | packaged by conda-forge | (default, Sep 8 2016, 14:22:31) [GCC 4.8.2 20140120 (Red Hat 4.8.2-15)]
• Python distribution/environment: Anaconda
• Operating System: Ubuntu Linux (Mint)
• Package version: 3.16.1

[wolph]

It appears you have stumbled upon 2 different but related bugs :)

1. The python utils library incorrectly tried to decode unicode (unicode is already decoded... that makes no sense). I've pushed a new release so your immediate fix is: pip install -U python-utils
2. The markers aren't converted to unicode so the character count is incorrect. A minor bug but perhaps a bit annoying ;)

I'll fix that as soon as possible

[bpmweel]

@wolph Thank you!

Since it now does not try to convert to unicode incorrectly, I can just pass a unicode string as a marker and the count is correct as well.

When passing an encoded unicode string it is still incorrect of course, but I don't see when you would actually use that. Although having the marker use the same semantics as the left and right here makes sense.

Cheers.

[wolph]

Both issues are fixed in version 3.17.0

[keriszafir]

bar.py line 55 - self.fd.write(line) causes a TypeError, because line is a byte string.
Perhaps line 54 should use converters.to_unicode rather than converters.to_str?

[wolph]

@elegantandrogyne Can you show some example code? Apparently the tests aren't covering your scenario.

[keriszafir]

The most basic example. Tested on python3.5 under Debian. progressbar version 3.17
>>>from time import sleep
>>>from progressbar import ProgressBar
>>>pbar = ProgressBar()
>>>for i in pbar(range(100)):
>>> sleep(0.1)