Hi @James_Parrott , @maje90 ,
Thanks for the suggestions. I will try out subprocess.check_output - I have not looked at that before.
I have attached here a simplified version of my actual pipeline (I think I have the main ‘flow’ implemented the same as my actual pipeline, which I think would be too confusing to repeat here in detail). In this flow, the user runs a subrprocess through a terminal to read in data from Excel (simulated in the example here with sleep(2)) then writes some data to csv file on disk. The second component reads in that CSV data and does some other work with it. In that scenario, I do get an error with the second GhPython component, since it tries to execute before the CSV file is fully written:
Runtime error (FileNotFoundException): Could not find file '/Users/em/Desktop/my_example.csv'.
Traceback:
line 2, in script
For reference, the GhPython component code is shown below:
Component 1:
import os
import subprocess
def run_subprocess_from_shell(commands):
# type: (list[str]) -> tuple[bytes, bytes]
"""Run a python subprocess.Popen THROUGH a MacOS terminal via a shell, using the supplied commands.
When talking to Excel on MacOS it is necessary to run through a Terminal since Rhino
cannot get the right 'permissions' to interact with Excel. This is a workaround and not
required on Windows OS.
Arguments:
----------
* _commands: (List[str]): A list of the commands to pass to Popen
Returns:
--------
* Tuple:
- [0]: stdout
- [1]: stderr
"""
# -- Create a new PYTHONHOME to avoid the Rhino-8 issues
CUSTOM_ENV = os.environ.copy()
CUSTOM_ENV["PYTHONHOME"] = ""
use_shell = True if os.name == "nt" else False
# -- Make sure the files are executable
shell_file = commands[0]
try:
subprocess.check_call(["chmod", "u+x", shell_file])
except Exception as e:
print("Failed to make the shell file executable: {}".format(e))
raise e
python_script_path = commands[3]
try:
subprocess.check_call(["chmod", "u+x", python_script_path])
except Exception as e:
print("Failed to make the python script executable: {}".format(e))
raise e
process = subprocess.Popen(
commands,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
shell=use_shell,
env=CUSTOM_ENV,
)
process.wait() # Ensure the process is completely finished
stdout, stderr = process.communicate()
return stdout, stderr
def run_on_macos(_shell_script, _python_3_script, _python_3_exe, _output_file):
# type: (str, str, str, str) -> str
"""Run the subprocess through a shell-script on MacOS
Runs in a 'terminal' in order to connect to Excel.
This is a workaround for the permissions issue on MacOS.
See:
https://discourse.mcneel.com/t/python-subprocess-permissions-error-on-mac-os-1743/142830/6
"""
execution_root = ""
# -- Build up the commands to run
commands = [
_shell_script, # -------------- The shell script to run
execution_root,
_python_3_exe, # ---------------- The python3-interpreter to use
_python_3_script, # ------------- The python3-script to run
_output_file, # ----------------- The output CSV filepath
]
stdout, stderr = run_subprocess_from_shell(commands)
return _output_file
def run(_shell_script, _python_3_script, _python_3_exe, _output_folder):
# type: (str, str, str, str) -> str
if os.name == "nt":
raise NotImplementedError("This component is not yet implemented for Windows.")
else:
output_file = os.path.join(_output_folder, "my_example.csv")
output_file = run_on_macos(_shell_script, _python_3_script, _python_3_exe, output_file)
return output_file
if _run:
result_csv_file_ = run(_shell_script, _python_3_script, _python_3_exe, _output_folder)
Component 2:
if _run:
with open(_result_csv_file) as f:
print "Running Step 2..."
I know the component 1 may look overly complicated for this simplified scenario, but in my actual pipeline the functions shown are used by several different components and are distributed over a couple different packages. I’ve tried to copy/paste the relevant bits here though.
thanks!!
@ed.p.may
Example Files:
that’s it! Thanks