Multiple progress bars appear at once in IntelliJ IDEA

[HatScripts]

Description

This obviously doesn't seem to be intended behaviour, but I'm not sure whether it's a problem with progressbar, with IntelliJ, or with my configuration.

Here's the full console output from running examples.py.

Code

Really any code that outputs a progress bar to the console, e.g.

import time
import progressbar

with progressbar.ProgressBar(max_value=10) as progress:
    for i in range(10):
        time.sleep(0.1)
        progress.update(i)

Versions

• Python version: 3.6.0 (v3.6.0:41df79263a11, Dec 23 2016, 07:18:10) [MSC v.1900 32 bit (Intel)]
• Python distribution/environment: IDLE
• Operating System: Windows 10
• Package version: 3.18.1

Edit:
I just tested the following simple function and the issue still occurs. Guess that means it's probably an issue with IntelliJ IDEA.

import sys
from time import sleep

def update_progress(i, total, length=10, fg="#", bg=" ", decimals=0):
    progress = 100 * (i / float(total))
    blocks = int(length * i // total)
    bar = fg * blocks + bg * (length - blocks)
    sys.stderr.write(f"\r[{bar}] {progress:.{decimals}f}%")
    sys.stderr.flush()

for i in range(0, 100):
    update_progress(i, 99)
    sleep(0.1)

[wolph]

I've done a few tests and unfortunately it appears to be a bug in IntelliJ IDEA

The simplest code you can run to test is roughly this:

import sys
import time


print('\nUsing stderr:')
for i in range(100):
    time.sleep(0.1)
    sys.stderr.write('\r%05d' % i)
    sys.stderr.flush()

print('\nUsing stdout:')
for i in range(100):
    time.sleep(0.1)
    sys.stdout.write('\r%05d' % i)
    sys.stdout.flush()

[HatScripts]

Indeed, it does appear to be a bug in IntelliJ IDEA.

I found these issue reports pertaining to it on JetBrains' issue tracker:

• Console treats carriage return ('\r') incorrectly
• Carriage return character is not printed in run console

If you're able to, please vote on them so that JetBrains can hopefully fix this.

[wolph]

I will, can't say I'm too hopeful about a quick fix unfortunately. My experience with the Jetbrains support is not too great. It generally takes years for them to fix issues.

[wolph]

To illustrate that point, the storing of ssh key passwords is broken (and some related features) for all of the Jetbrains suite and has been since 2009 or perhaps even sooner: https://youtrack.jetbrains.com/issue/IDEA-24944#tab=Linked%20Issues

That's a bug with "Major" priority and has been reported 8 separate times.

[wolph]

It's actually even more broken than I thought (and has been for years it seems):

The issue can be mitigated slightly be running the script in debug mode but it doesn't solve it

[HatScripts]

@wolph Wow, you're right. Just tested it with Java as well to confirm it's not somehow Python-specific, and got the same kinds of results, where stdout and stderr are interweaved.

Because I've only ever used IntelliJ as my IDE, I always thought this is how stdout and stderr were naturally supposed to behave (at least in Java). I thought the order they appear in the console wouldn't necessarily correlate to the order the print methods are called in, because it seems there is a thread which handles stdout and a thread which handles stderr.

However, when I compile and run the same program from the command line, I get the output of 0 to 19, in the correct order, for both Java and Python. In retrospect this makes a lot more sense and this clearly is a bug with IntelliJ.

But is this the same bug that causes printing \r to give unexpected results? You don't need to print to both streams to get weird results from \r:

import sys
import time

for i in range(100):
    time.sleep(0.1)
    sys.stdout.write('\r%05d' % i)
    sys.stdout.flush()

[wolph]

It's a different bug but it could be related. The Jetbrains console code is obviously quite broken and I'm not too hopeful that they'll fix it any time soon :(

I just found another bug report from 2 years ago: https://youtrack.jetbrains.com/issue/PY-17489

[wolph]

Since this is a bug in jetbrains which I can't fix... I'll have to close it.

[fireundubh]

I don't see this issue in PyCharm 2017.3.2 with:

with progressbar.ProgressBar(max_value=len(files)) as bar:
	for file in bar(files):
		# do stuff
		bar.update(bar.value)

Only oddity is the progress string is printed again on completion:

D:\dev\Python364\python.exe -m tools.update_json_with_text
Processing files...
100% (1397 of 1397) |#####################| Elapsed Time: 0:00:44 Time: 0:00:44
100% (1397 of 1397) |#####################| Elapsed Time: 0:00:44 Time: 0:00:44

Process finished with exit code 0

[wolph]

Since it's printed twice I'm guessing the issue still exists, but that's not to say the issue hasn't been improved.

I'll do some testing :)

[wolph]

It still looks quite broken to me, but they've decreased the update interval I think