Python3.5 subprocess not writing on terminal while writing print logs in file

December 26, 2017 622 views
Python Logging CentOS

Below is my script which calls another script using below example and produces entire output after execution is completed while expectation is that it should produce live output as well as write in file at the same time.

My_Tests.py :

from subprocess import Popen, PIPE, STDOUT
with Popen("./tests.py", stdout=PIPE, stderr=STDOUT, bufsize=1, universal_newlines=True) as p, \  
    open('tests.log', 'ab') as file:  
    for line in p.stdout: # b'\n'-separated lines  
        print(line, end='') #new addition  
        sys.stdout.buffer.write(line)  
        file.write(line)

Results:

./My_Tests.py ALL
TC 1 : PASSED
TC 2 : PASSED
Tatal Passed Tests : 2
Tatal Failed Tests : 0 

Issue: Execution of script is not producing output line by line and printing entire output in the end while expectation is terminal logging and saving logs in file shall happen simultaniously.

1 Answer

You may want to question your expectations.

Popen() will not call "tests.py" to run and then continue with the execution of "My_Tests.py".
This is called "blocking", what you want is a "non-blocking" way to call a new process from a python script.

From the "subprocess" documentation (https://docs.python.org/3/library/subprocess.html):
Run the command described by args. Wait for command to complete, then return a CompletedProcess instance.

Have another answer? Share your knowledge.